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ABSTRACT 


Efficiency  in  the  acquisition  of  major  defense  systems  is 
of  paramount  importance,  given  today's  environment  of 
declining  resources.  To  achieve  this  efficiency,  the  project 
office,  as  the  responsible  agency  for  such  acquisitions,  must 
perform  its  designated  functions  both  efficiently  and 
effectively.  The  two  most  important  indicators  reflecting  the 
efficiency  and  effectiveness  of  the  project  office  are  program 
health  and  customer  satisfaction.  The  objective  of  this 
research  was  to  develop  an  automated  self-assessment 
instrument  for  use  by  the  military  project  manager  to  measure 
the  program  health  and  customer  satisfaction  of  his/her 
organization.  This  thesis  provides  the  project  manager  with 
a  viable  model  depicting  the  critical  organizational  design 
factors  impacting  the  program  health  and  internal  customer 
satisfaction  of  the  military  project  office.  This  study  also 
provides  the  project  manager  a  pilot  implementing  instrument 
with  which  to  assess  the  program  health  and  internal  customer 
satisfaction  of  his/her  organization.  Finally,  this  research 
effort  has  produced  an  independent  software  application, 
specifically  designed  to  automate  the  self-assessment  process 
within  a  military  project  management  office. 
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I. 


INTRODUCTION 


A.  PURPOSE 

The  purpose  of  this  thesis  is  to  develop  a  unit  self- 
assessment  model  and  a  software-based  survey  instrument  to 
effectively  measure  the  program  health  and  customer 
satisfaction  of  a  military  project  management  office. 

B.  BACKGROUND 

In  July  1994,  the  Program  Executive  Office  (PEO) , 
Tactical  Missiles,  U.S.  Army  Material  Command  (AMC)  directed 
all  project  managers  within  the  PEO  to  conduct  a  unit  self- 
assessment  using  the  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction,  based  on  the 
Presidential  Quality  Award  criteria.  After  conducting  these 
assessments,  two  of  the  Project  Managers  (PMs) ,  PM  Close 
Combat  Anti-armor  Weapon  Systems  (CCAWS)  and  PM  Air-to-Ground 
Missile  Systems  (AGMS) ,  conducted  internal  reviews  of  the 
Presidential  Award  process  and  criteria  and  their 
effectiveness  in  measuring  the  critical  factors  which 
determine  the  overall  health  of  a  project  office  and  the  level 
of  its  customers'  satisfaction.  These  independent  reviews 
suggested  that  the  Presidential  Award  criteria  and  its 
implementing  instrument,  the  Self-assessment  Guide  for 
Organizational  Performance  and  Customer  Satisfaction,  were 
inappropriate  to  accurately  assess  and  track  the  level  of 
program  health  and  customer  satisfaction  within  a  project 
management  office. 

Based  on  these  findings,  the  CCAWS  and  AGMS  Project 
Managers  requested  that  the  Naval  Postgraduate  School  develop 
an  alternative  assessment  instrument  that  will  effectively 
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assess  the  factors  which  directly  contribute  to  the  program 
health  and  customer  satisfaction  of  a  project  management 
office . 

C.  THESIS  OBJECTIVES 

The  objectives  of  this  thesis  are  to  develop: 

1.  A  unit  assessment  model  and  a  pilot  survey 
instrument  that  are  appropriate  and  effective  for  measuring 
the  program  health  and  customer  satisfaction  of  a  project 
management  office. 

2.  An  automated,  network-based  user  interface,  data 
storage,  analysis,  and  presentation  application  that 
simplifies  the  unit  self-assessment  process. 


D .  ASSUMPTIONS 

This  researcher  made  the  following  assumptions  concerning 
this  research  effort: 

1.  The  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction  is  an  inappropriate 
instrument  to  measure  the  program  health  and  customer 
satisfaction  within  a  project  management  office,  for  the 
reasons  identified  by  the  sponsoring  organizations. 

2.  The  assessment  instrument  developed  as  an 
alternative  to  the  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction  is  only  intended  for  use 
in  a  military  project  management  office. 

3.  The  factors  contributing  to  program  health  and 
customer  satisfaction  are  consistent  across  project  management 
offices . 

4.  The  sponsoring  organizations  do  not  intend  to 
significantly  modify  their  computer  operating  systems  within 
the  next  five  year  period. 


E .  RESEARCH  QUESTIONS 

1 .  Primary 

Does  the  proposed  unit  self-assessment  package  satisfy 
the  sponsors'  requirement  to  efficiently  and  effectively 
assess  and  track  program  health  and  customer  satisfaction? 

2 .  Subsidiary 

a.  Does  the  proposed  assessment  address  and  correct 
the  deficiencies  of  the  Self-assessment  Guide  for 
Organizational  Performance  and  Customer  Satisfaction, 
identified  by  the  sponsors'  management  teams  and  users? 

(1)  Are  the  assessment  items  applicable  to  a 
project  management  office? 

(2)  Is  the  wording  or  content  of  the  assess¬ 
ment  items  confusing  to  the  intended  users? 

(3)  Is  the  assessment  too  long  and  time 

consuming? 

(4)  Are  the  assessment  items  appropriate  to 
solicit  responses  from  all  personnel  within  the  project 
office? 


(5)  Does  the  assessment  solicit  both  quant¬ 
itative  and  qualitative  feedback? 

b.  Is  the  proposed  software  application  an 
effective  method  to  administer  the  recommended  assessment? 

(1)  Is  the  software  application  compatible 
with  the  existing  computer  resources  used  by  the  CCAWS  and 
AGMS  project  management  offices? 

(2)  Is  the  software  user  friendly? 

(3)  Is  the  software  application  accessible  to 
its  intended  users? 


(4)  Does  the  software  enhance  the  overall 
conduct  of  the  proposed  assessment? 
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(5)  Does  the  software  simplify  the  user's 
ability  to  provide  quantitative  and  qualitative  feedback? 

(6)  Does  the  software  simplify  the  facili¬ 
tator's  ability  to  collect,  compile,  analyze,  and  present 
generated  feedback? 

c.  Does  the  proposed  unit  self-assessment  package 
provide  project  managers  with  the  information  and  tools 
necessary  to  monitor  and  improve  program  health  and  customer 
satisfaction? 

(1)  Is  the  package  capable  of  compiling  and 
analyzing  data  generated  from  responses  in  a  manner  that 
provides  useful  feedback? 

(2)  Are  the  types  and  depth  of  analyses 
included  in  the  package  sufficient  to  meet  the  managers' 
requirements? 

(3)  Is  the  package  capable  of  presenting  the 
results  of  generated  feedback  and  analyses  in  a  clear  and 
useful  manner? 

(4)  Is  the  package  capable  of  storing  inform¬ 
ation,  for  the  purpose  of  comparison,  for  a  period  of  not  less 
than  five  (5)  years? 

(5)  Does  the  package  provide  project  managers 
with  the  tools  necessary  to  monitor  the  improvement  of 
identified  weaknesses? 

F.  ORGANIZATION  OF  THESIS 

Chapter  II  of  this  study  details  the  methodology  employed 
by  this  researcher.  It  begins  by  documenting  efforts  to 
identify  an  appropriate  candidate  assessment  instrument.  It 
then  describes  the  software  design  techniques  used  in  the 
prototype  software  development.  The  chapter  concludes  with  a 
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description  of  the  quantitative  methods  used  to  evaluate  the 
candidate  unit  self-assessment  and  the  prototype  software 
application. 

Chapter  III  provides  background  information  on  the 
military  project  management  office.  It  begins  with  a 
description  of  the  military  project  management  environment. 
The  chapter  continues  with  a  discussion  of  the  major 
functional  areas  common  to  most  project  management  offices. 
This  chapter  concludes  by  introducing  the  CCAWS  and  AGMS 
project  offices,  the  sponsors'  of  this  thesis. 

Chapter  IV  addresses  the  assessment  instrument.  This 
chapter  begins  with  a  discussion  of  the  theory  and  general 
framework  underlying  the  development  and  implementation  of 
assessment  instruments.  The  chapter  then  presents  a 
theoretical  model  of  organizational  dynamics  developed  by  the 
Harvard  School  of  Business.  This  model  is  used  as  the  basis 
to  compare  the  organizational  models  that  support  the 
assessment  instruments  addressed  in  this  study.  Next,  the 
chapter  describes  and  evaluates  the  two  models  and  assessment 
instruments  considered  in  this  study.  Discussed  first  is  the 
Presidential  Award  Model  and  the  Self-assessment  Guide  for 
Organizational  Performance  and  Customer  Satisfaction,  used  by 
the  CCAWS  and  AGMS  Project  management  Offices  in  July  1994. 
This  is  followed  by  a  discussion  of  the  Strategic  Leadership 
Planning  Model  and  its  assessment  instrument,  the  United 
States  Army,  Europe  (USAREUR)  Organizational  Diagnostic 
Survey.  This  model  and  assessment  instrument  were  selected  by 
this  researcher  as  suitable  alternatives  to  the  Presidential 
Award  Model  and  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction.  Also  addressed  are 
modifications  to  the  original  USAREUR  Organizational 
Diagnostic  Survey  prior  to  its  evaluation. 
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Chapter  V  focuses  on  the  automation  of  the  self- 
assessment  process.  It  begins  with  a  description  and 
evaluation  of  the  Continuous  Improvement  (Cl)  Toolkit  for 
Windows,  a  commercial  software  package  initially  selected  by 
this  researcher  as  a  solution  to  automate  the  self-assessment 
process.  Next,  the  chapter  describes  the  development  of  an 
independent  prototype  software  program  to  serve  as  the  user 
interface  for  the  collection  of  assessment  data.  The  chapter 
concludes  with  a  description  of  the  database  storage, 
analysis,  and  presentation  applications  designed  using 
Microsoft  Access. 

Chapter  VI  begins  by  presenting  the  data  and  qualitative 
responses  accumulated  about  the  candidate  unit  self- 
assessment,  assessment  items,  and  the  proposed  software 
application  acquired  during  the  prototype  testing.  .  The 
chapter  then  presents  an  analysis  and  discussion  of  the  issues 
related  to  the  Strategic  Leadership  Planning  Model,  the 
candidate  self-assessment  instrument,  and  the  assessment  items 
identified  during  the  assessment  evaluation.  This  chapter 
concludes  with  an  analysis  of  the  prototype  assessment 
software  application  and  a  discussion  of  anomalies  discovered 
in  the  software  during  the  prototype  testing. 

Chapter  VII  presents  conclusions  from  the  data  analysis 
and  discussion  and  makes  recommendations  for  implementation  to 
the  CCAWS  and  AGMS  Project  Managers.  This  chapter  concludes 
with  recommendations  for  further  study. 
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II.  METHODOLOGY 


A.  INTRODUCTION 

This  chapter  explains  the  methods  employed  in  the 
development  and  analysis  of  the  proposed  unit  self-assessment 
package.  It  begins  with  a  description  of  the  qualitative 
methods  used  to  define  the  research  requirements  and  to  obtain 
background  information  pertaining  to  the  organizational 
assessment  process,  the  military  project  management  office, 
and  available  assessment  instruments  and  quality  improvement 
software.  The  chapter  then  focuses  on  the  techniques  employed 
in  the  development  of  the  prototype  user  interface,  data 
storage,  analysis,  and  presentation  software  applications. 
Next,  the  chapter  describes  the  quantitative  methods  used  to 
evaluate  the  candidate  self-assessment  and  the  prototype 
software  applications.  This  chapter  concludes  with  a 
description  of  the  limitations  applicable  to  this  research. 

B.  QUALITATIVE  METHODS 

1 .  Interviews 

Research  began  with  the  identification  of  specific 
research  requirements  and  the  justification  for  the  research 
effort.  This  was  accomplished  by  conducting  interviews  with 
the  CCAWS  Project  Manager,  the  AGMS  Deputy  Project  Manager, 
and  personnel  from  within  both  project  management  offices, 
identified  as  participants  of  the  July  1994  unit  assessments 
using  the  Self-assessment  Guide  for  Organizational  Performance 
and  Customer  Satisfaction. 

2 .  Archival  Research 

Once  the  requirements  were  defined,  a  literature  review 
was  conducted  of  available  materials  to  locate  existing 
assessment  instruments  and  quality  improvement  software 
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programs  within  the  public  and  private  sectors  with  the 
potential  for  satisfying  the  sponsors'  requirements.  A  second 
literature  review  was  conducted  to  ascertain  background 
information  pertaining  to  the  research  effort.  This  review 
focused  on  the  rationale  for  conducting  unit  self-assessments 
and  the  organizational  dynamics  of  a  military  project 
management  office. 

3.  Participant  Observation 

In  addition  to  conducting  literature  reviews,  this 
researcher  observed  first-hand  the  CCAWS  and  AGMS  project 
offices  on  several  occasions  totaling  a  period  of  one  month. 
These  visits  verified  the  amount  and  types  of  computer 
resources  currently  in  use  by  the  project  offices,  to  include 
identifying  potential  hardware  and  software  compatibility 
issues.  Also  observed  were  the  functions  of  a  military 
project  management  office,  its  environment,  and  the 
interaction  between  its  divisions,  branches,  and  product 
lines . 

4 .  Researcher  Educational  Preparation 

In  February  1995,  this  researcher  attended  a  four-day 
workshop  on  the  Continuous  Improvement  (Cl)  Toolkit  for 
Windows,  a  potential  solution  to  automate  the  unit  self- 
assessment  process.  The  workshop  was  designed  to  educate 
system  administrators  and  assessment  facilitators  in  the 
installation,  setup,  modification,  capabilities,  and 
procedures  of  the  Cl  Toolkit. 

In  addition  to  the  Cl  Toolkit,  the  above  workshop  focused 
on  the  organizational  assessment  process.  Specifically,  the 
workshop  addressed  the  Malcolm  Baldrige  Award  Criteria, 
preparation  and  submission  of  a  Baldrige  application,  and  the 
integration  of  the  Cl  Toolkit  into  the  Baldrige  submission 
process . 
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This  researcher  also  attended,  as  part  of  the  Master's 
degree  requirements,  a  course  in  strategic  management.  The 
primary  focus  of  this  course  was  the  identification  and 
analysis  of  organizational  design  factors.  Specifically,  the 
course  presented  a  theoretical  model,  developed  by  the  Harvard 
School  of  Business  and  modified  by  Dr.  Nancy  Roberts, 
depicting  the  interaction  between  various  inputs,  organization 
design  factors,  and  their  resultant  outcomes.  Knowledge 
acquired  in  this  course  was  applied  in  the  evaluation  and 
comparison  of  the  organizational  models  that  support  the 
assessment  instruments  addressed  in  this  study. 

5.  Model  Construction 

Using  the  model  developed  by  the  Harvard  School  of 
Business  as  a  basis  for  comparison,  this  researcher  evaluated 
the  organizational  models  supporting  the  Self-assessment  Guide 
for  Organizational  Performance  and  Customer  Satisfaction  and 
the  USAREUR  Organizational  Diagnostic  Survey,  the  candidate 
assessment  instrument  selected  in  this  study  as  a  viable 
alternative  to  the  aforementioned  Self-assessment  Guide  for 
Organizational  Performance  and  Customer  Satisfaction.  The 
results  of  these  comparisons  were  then  used  as  the  basis  to 
create  a  prototype  model  that  more  accurately  reflects  the 
critical  organizational  design  factors  of  a  military  project 
management  office. 

6.  Software  Development 

Applying  the  information  gained  in  interviews  with  CCAWS 
and  AGMS  management  personnel,  an  integrated  software  program 
designed  to  satisfy  the  sponsors'  specific  automation 
requirements  was  developed.  This  program  consists  of  two 
distinctive  applications:  a  user  interface  and  a  data  storage, 
analysis,  and  presentation  application. 
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The  user  interface  application  was  developed  using 
Novell's  AppWare,  a  graphical,  object  oriented  design  and 
development  tool.  The  interface  was  designed  to  satisfy  four 
of  the  nine  sponsor  automation  requirements.  These 
requirements  are: 

•  Enable  the  project  office  to  administer  an 
organizational  assessment  over  an  existing  network 
structure . 

•  Provide  a  familiar,  user  friendly  interface  to  the 
assessment  participants  for  the  purpose  of  data 
collection. 

•  Compatibility  with  existing  computer  resources  and 
software  programs. 

•  Export  collected  data  to  the  analysis  and 
presentation  application. 

The  analysis  and  presentation  application  was  developed 
using  Microsoft's  Access  database  program.  The  application 
was  designed  to  address  the  remaining  five  user  automation 
requirements : 

•  Import  collected  data  from  the  user  interface 
program. 

•  Analyze  collected  data. 

•  Store  collected  data  for  a  five  year  period. 

•  Present  analyzed  data  in  a  meaningful  format. 

•  Enable  data,  analysis,  and  presentation  output  to 
either  a  monitor  or  printer. 

C.  QUANTITATIVE  METHODS 

1 .  Prototype  Evaluation 

A  sample  unit  self-assessment  of  both  the  CCAWS  and  AGMS 
project  management  offices  was  conducted  in  June  1995  for  the 
purpose  of  evaluating  the  candidate  assessment  and  the 
prototype  software  application.  The  evaluation  was 
specifically  designed  to  evaluate  the  appropriateness  of  the 
individual  functional  areas  and  items  of  the  USAREUR 
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Organizational  Diagnostic  Survey  (modified)  for  use  in 
measuring  the  program  health  and  customer  satisfaction  of  a 
project  management  office  and  to  verify  the  structure, 
functionality,  and  effectiveness  of  the  prototype  software 
application  (user  interface)  to  collect  user  feedback.  Data 
and  feedback  from  the  prototype  evaluation  were  accumulated 
through  the  use  of  multiple  surveys,  discussed  below,  and 
selected  project  manager  and  unit  facilitator  interviews. 

2.  Survey  Administration 

Responses  were  elicited  as  to  the  appropriateness  of  the 
candidate  unit  self-assessment  and  the  prototype  software 
application  through  the  use  of  four  independent  surveys.  Each 
of  the  four  surveys  were  completed  by  the  participants  of  the 
candidate  assessment  evaluation  and  prototype  software 
testing.  The  sample  used  in  these  surveys  and  the  design  of 
each  survey  are  discussed  below, 
a.  The  Sample 

Each  of  the  four  surveys  were  administered  to 
personnel  participating  in  the  prototype  assessment  and 
software  evaluation,  representing  the  sponsoring  organizations 
(CCAWS  and  AGMS) .  Every  effort  was  made  to  obtain  a  sample 
that  accurately  represents  the  demographic  makeup  of  each 
participating  organization.  Within  these  guidelines,  the 
selection  of  participating  individuals  was  determined  by 
random  draw. 

(1)  Population.  The  population  from  which  the 
sample  was  drawn  consisted  of  all  personnel  assigned  or 
attached  to  either  the  CCAWS  of  AGMS  project  offices.  The 
AGMS  project  office  employs  97  personnel,  divided  into  four 
division  and  17  separate  branch  offices.  CCAWS  employs  a 
total  of  103  personnel  in  four  divisions,  separated  into  15 
branch  offices.  The  total  population  numbered  200  personnel. 
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This  population  was  divided  by  the  researcher  into  three  major 
demographic  categories:  level  of  responsibility,  sex,  and 
employment  status.  The  level  of  responsibility  category  was 
sub-divided  into  management  personnel,  clerical  personnel,  and 
all  others.  Sex  was  further  divided  into  male  and  female 
participants,  and  employment  status  was  broken  down  into 
military  personnel  and  civilian  employees.  The  demographic 
break-down  of  the  population  is  presented  in  Table  1. 


MGT 

Clerical 

Other 

Male 

Female 

CIV 

AGMS 

12 

n 

74 

65 

32 

n 

89 

CCAWS 

23 

14 

66 

71 

32 

9 

94 

Total 

35 

25 

140 

136 

64 

17 

183 

200 


Pop.  % 

18% 

12% 

70% 

68% 

32% 

9% 

91% 

100% 

Table  1:  The  Population. 

(2)  The  Sample.  The  sample  target  was  50 
participants,  25%  of  the  total  population,  consisting  of  an 
equal  percentage  from  each  demographic  category  defined  above 
and  a  proportionate  number  from  each  of  the  participating 
organizations.  The  demographic  make-up  of  the  actual  sample, 
by  total  and  percentage,  is  presented  in  Table  2. 

The  accumulated  sample  is  representative  of  the 
CCAWS/AGMS  population,  with  the  following  exception:  The 
sample  contains  a  disproportionate  number  of  military 
managers.  This  is  due  to  unavoidable  scheduling  conflicts  and 
the  researcher's  decision  to  select  alternates  from  within 
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these  categories,  when  available,  to  maximize  input  from  these 
categories  with  respect  to  Survey  #2. 


MGT 

Clerical 

Other 

Male 

Female 

MIL 

CIV 

AGMS 

3 

3 

19 

17 

8 

2 

23 

CCAWS 

9 

3 

13 

14 

7 

5 

20 

Total 

12 

6 

32 

35 

15 

7 

43 

50 

|  Sample  %  jj 

24% 

12% 

64% 

70% 

30% 

1  14% 

86% 

100% 


Table  2:  The  Sample. 

b.  Survey  #1 

Survey  #1  (Appendix  A)  is  the  USAREUR  Organizational 
Diagnostic  Survey  (modified) .  Researcher  modifications  to  the 
original  USAREUR  Organizational  Diagnostic  Survey  (Appendix  B) 
preceding  the  assessment  evaluation  and  prototype  testing  are 
discussed  below.  This  survey,  in  its  final  form,  upon 
completion  of  the  researcher's  analysis  is  the  CCAWS/AGMS  Unit 
Self-assessment  Survey  (Appendix  C) ,  a  deliverable  product  of 
this  thesis. 

(1)  Survey  Design.  The  USAREUR  Organizational 
Diagnostic  Survey  (modified)  is  composed  of  seventy- four  (74) 
items,  grouped  into  eight  functional  areas  and  a  general 
category.  The  functional  areas  represented  in  the  survey  are: 

•  Strategy 

•  Tasks 

•  Structure 

•  People 
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•  Rewards 

•  Processes 

•  Leadership 

•  Technology 

This  survey  was  designed  to  measure  each  respondent's  opinion 
as  to  the  program  health  of  his/her  respective  organization, 
with  respect  to  each  functional  listed  above. 

(2)  Survey  Modifications.  Several  mod¬ 
ifications  to  the  original  USAREUR  Organizational  Diagnostic 
Survey  were  made  prior  to  conducting  the  assessment  evaluation 
and  prototype  testing.  These  modifications  were  made  to 
accomplish  the  following  items: 

•  Separate  items  soliciting  multiple  responses  into 

individual  items . 

•  Clarify  items  with  complex  or  ambiguous  wording. 

•  Remove  items  clearly  inappropriate  for  use  in  a 

project  management  office. 

•  Replace  all  references  to  the  "organization"  with 

the  "project  management  office". 

(3)  Analysis.  The  analysis  of  data  measuring 
the  health  of  the  participating  organizations  is  beyond  the 
scope  of  this  thesis  and  therefore  will  not  be  presented  as 
part  of  this  study.  These  data  elements  were  collected  solely 
for  the  purposes  of  establishing  an  initial  baseline  to  be 
used  by  each  participating  unit  for  future  assessments  and  to 
accumulate  realistic  data  for  use  in  the  design  of  data 
queries,  analysis  tools,  and  presentation  layouts. 

c .  Survey  #2 

Survey  #2  is  the  Unit  Self-assessment  Item  Survey. 
(Appendix  D) 

(1)  Survey  Design.  Survey  #2  was  designed  to 
solicit  participant  feedback  about  each  individual  item 
incorporated  in  the  USAREUR  Organizational  Diagnostic  Survey, 
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as  modified  by  this  researcher.  Respondents  were  asked  to 
rate  six  different  aspects  measuring  three  general  categories 
of  each  item.  The  three  categories  were: 

•  The  wording  of  the  item 

•  The  item' s  content 

•  The  scale  provided  to  answer  the  item 

With  respect  to  the  item' s  wording,  respondents  were  asked  to 
rate  the  wording's  clarity  and  conciseness.  As  for  the  item's 
content,  respondents  were  asked  to  rate  the  understandability 
of  the  item,  the  relevancy  of  the  content  to  their  job 
position,  and  the  suitability  of  the  item.  In  this  context 
suitability  is  defined  as  unoffensive.  Lastly,  each 
respondent  was  asked  to  rate  the  appropriateness  of  the  scale 
provided  to  respond  to  the  item. 

(2)  Analysis.  Respondents  were  asked  to  rate 
each  aspect  described  above  using  a  bipolar  scale  with  three 
intermediate  points,  five  points  total  (Figure  1)  .  The 
endpoints  of  each  scale  were  labeled  with  opposing  adjective 
descriptors  connoting  a  positive  or  negative  response.  Each 
respondent  was  also  asked  to  provide  comments  in  these  areas 
on  each  item. 

Is  the  item's  wording: 

□  □  □  □  □ 

I - 1 - 1 - 1 - ! 

Ambiguous  Clear 

Figure  1:  Example  Bipolar  Survey  Item. 


After  all  data  were  collected,  individual 
responses  were  then  translated  to  numerical  values  on  a  scale 
from  negative  two  to  two,  zero  being  a  neutral  response.  The 
numerical  values  were  then  grouped  by  item  and  aspect. 
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Calculations  were  performed  to  determine  percentage  response 
of  the  maximum  possible  score  (100)  per  item,  and  the  mean 
response  and  standard  deviation  for  each  item,  by  aspect. 
Items  were  then  reviewed  using  the  percentage  of  the  maximum 
possible  score  calculations  to  identify  questionable  items. 
The  screening  criteria  for  this  review  was  99  percent.  Items 
identified  as  questionable  were  then  analyzed  with  respect  to 
each  measured  aspect  to  identify  those  aspects  containing 
significant  shortcomings.  A  significant  shortcoming  was 
defined  as  an  item  aspect  receiving  less  than  a  mean  score  of 
1.91,  which  corresponds  to  an  effectiveness  rating  of  less 
than  95%.  Items  with  one  or  more  significant  shortcoming  were 
then  analyzed  against  the  qualitative  responses  for  each 
deficient  aspect  to  determine  the  appropriate  corrective 
action. 

d.  Survey  #3 

Survey  #3  is  the  Unit  Self-assessment  Survey,  listed 
as  Part  A  of  the  Unit  Self-assessment  and  Software  Survey. 
(Appendix  E) 

(1)  Survey  Design.  Survey  #3  was  designed  to 
elicit  data  and  qualitative  feedback  on  the  candidate  self- 
assessment's  usefulness,  applicability,  and  procedures,  as 
well  as  general  comments  and  suggestions  for  improvement. 
This  survey  employed  two  different  questioning  techniques:  the 
bipolar  method,  described  in  survey  #2,  and  open  items 
eliciting  a  qualitative  text  or  numerical  response. 

(2)  Analysis.  Bipolar  items,  as  in  survey  #2, 
asked  respondents  to  rate  each  aspect  described  above  of  each 
item  using  a  scale  with  three  intermediate  points.  Again,  the 
endpoints  of  each  scale  were  labeled  with  opposing  adjective 
descriptors  connoting  a  positive  or  negative  response. 
Responses  to  these  items  were  analyzed  in  a  similar  manner  as 
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those  described  in  survey  #2.  The  difference  being  that  there 
was  no  requirement  to  group  individual  items  to  conduct  an 
initial  screening  for  potential  weaknesses. 

Open  items  were  designed  to  enable  each 
respondent  to  provide  his/her  opinion,  comments,  or 
suggestions  concerning  the  candidate  assessment  and  its 
procedures.  Responses  to  these  items  were  summarized  for  use 
in  the  analysis  and  discussion  of  the  overall  assessment 
process . 

e .  Survey  #4 

Survey  #4  is  the  Unit  Self-assessment  Software 
Survey,  listed  as  Part  B  of  the  Unit  Self-assessment  and 
Software  Survey.  (Appendix  E) 

(1)  Survey  Design.  Survey  #4  was  designed  to 
solicit  feedback  on  the  self-assessment  software's  usefulness, 
applicability,  and  procedures,  as  well  as  general  comments  and 
suggestions  for  improvement  of  the  software.  As  with  Survey 
#3,  this  survey  employed  two  different  questioning  techniques: 
the  bipolar  method,  and  open  items  eliciting  a  qualitative 
response . 

(2)  Analysis.  The  analysis  of  the  items  in 
this  survey  is  identical  to  that  described  for  survey  #3. 

D.  LIMITATIONS 

The  following  limitations  apply  to  this  research  effort: 

1.  This  thesis  is  limited  to  the  study  of  an  assessment 
instrument  for  use  in  a  military  project  management  office. 

2.  The  data  and  qualitative  responses  used  in  the 
analysis  of  the  candidate  assessment  instrument  and  the 
prototype  software  application  are  limited  to  inputs  from  the 
CCAWS  and  AGMS  project  management  offices. 
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3.  Software  applications  developed  as  the  result  of  this 
study  are  the  sole  property  of  the  United  States  Government 
and  can  only  be  modified  with  express  permission  of  the  CCAWS 
or  AGMS  Project  Managers. 

4.  Modifications  to  software  developed  using  Novell's 
AppWare  requires  the  purchase  of  the  AppWare  development 
software  and  obtaining  application  project  files  that  are 
external  to  the  program.  These  project  files  have  been 
retained  by  the  researcher  and  the  CCAWS  and  AGMS  project 
management  offices. 

E.  SUMMARY 

The  purpose  of  this  research  effort  was  to  develop  an 
automated  unit  self-assessment  package  appropriate  to  measure 
the  program  health  and  customer  satisfaction  of  a  military 
project  management  office.  Using  both  qualitative  and 
quantitative  research  techniques,  this  researcher  has 
evaluated  the  unit  assessment  previously  used  by  the  CCAWS  and 
AGMS  project  management  offices  against  user  specific 
requirements;  identified  the  shortcomings  with  the  previous 
assessment;  selected  an  existing  alternative  as  a  candidate 
assessment;  tested,  analyzed,  and  modified  the  candidate 
assessment;  automated  the  assessment  process;  and  provided 
recommendations  the  CCAWS  and  AGMS  Project  Managers  for  the 
implementation  of  the  proposed  assessment  and  software 
package . 
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III.  THE  PROJECT  MANAGEMENT  OFFICE 


A.  INTRODUCTION 

A  Project  Management  Office  (PMO)  is  an  organization 
responsible  to  facilitate  the  design,  development,  production, 
fielding,  and  support  of  a  major  systems  acquisition.  The 
purpose  of  this  chapter  is  to  provide  background  information 
about  the  military  project  office  as  a  customer  of  the  unit 
self-assessment.  This  discussion  is  divided  into  four  areas: 
The  military  project  management  environment;  the  project 
office  structure;  functions  of  the  project  office;  and  a 
description  of  the  CCAWS  and  AGMS  project  offices. 


B.  ENVIRONMENT 

The  environment  in  which  the  PMO  exists  is  quite  complex. 
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Within  this  environment,  depicted  in  Figure  2,  are  many 
agencies,  both  within  and  external  to  the  Department  of 
Defense,  with  which  personnel  within  the  PMO  must  directly 
interact  with  on  a  daily,  weekly,  or  monthly  basis  to  ensure 
the  successful  accomplishment  of  the  project's  mission.  In 
addition  to  their  coordinating  role,  many  of  these  agencies 
provide  guidance  or  exert  a  significant  amount  of  control  over 
the  project  office's  objectives  and  activities.  The  project 
office  must  proactively  respond  to  the  needs  and  desires  of 
these  agencies  for  a  number  of  varying  reasons.  These  reasons 
include : 

•  Congressional  support. 

•  Public  support. 

•  Resources. 

•  Legal  Support. 

•  Task  accomplishment. 


C .  STRUCTURE 

The  military  project  management  office  is  structured 
along  the  traditional  functional  and  commodity  area  lines. 
Many  project  offices  operate,  however,  along  product  lines  in 
Integrated  Product  Teams  (IPTs)  to  enhance  the  efficiency  and 
effectiveness  of  the  services  they  perform.  Viewing  the 
project  office's  structure  as  it  relates  to  the  implementation 
of  its  IPTs,  the  project  office  becomes  a  matrixed 
organization  (Figure  3) .  Under  this  concept,  selected 
personnel  assigned  within  the  organization's  functional  or 
commodity  divisions  are  aligned  to  provide  dedicated  support 
in  their  specialty  towards  the  development  of  one  or  more  of 
the  project  office's  specified  products.  IPTs  are  normally 
lead  by  the  manager  with  responsibility  for  the  completed 
product.  Members  of  the  IPT  remain  assigned  in  their 
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functional  or  commodity  area,  but  are  responsible  to  the  IPT 
team  leader  for  product  related  activities. 


Figure  3:  Example  Matrix  Organization. 


D .  FUNCTIONS 

Each  military  project  office  is  uniquely  structured  and 
differs  in  its  approach  in  supporting  the  development, 
production,  and  fielding  of  its  assigned  weapon  system. 
Despite  these  differences,  all  military  project  offices 
perform  the  same  general  or  "core"  functions .  These  functions 
and  their  integration  with  each  other  are  depicted  in  Figure 
4.  (Cleland,  Gallagher,  and  Whitehead,  1993,  p.  10.21)  The 
"core"  functions  identified  by  these  authors  include: 
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•  Project  Management 

•  Contract  Management 

•  Business  Management 

•  System  Engineering  Management 

•  Integrated  Logistic  Support  Management 

•  Production  Management 

•  Configuration  Management 

•  Risk  Management 


Figure  4:  Project  Management  Functional  Integration. 


Project  Management  is  the  function  of  integrating  each  of  the 
projects  subordinate  functions  for  the  successful  development 
of  the  desired  system.  This  integration  is  facilitated 
through  the  business,  administrative,  and  technical  management 
of  the  sequence  of  project  activities  in  the  project  cycle. 
Contract  Management  or  acquisition  management  is  a  project 
function  dedicated  to  the  planning,  design,  implementation, 
and  administration  aspects  of  contractual  obligations  and 
documents  involved  in  the  acquisition  of  a  major  system. 
Tasks  included  in  this  function  include: 

•  Procurement  planning  and  guidance. 
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•  Developing  contract  requirements. 

•  Contract  administration. 

Business  Management  encompasses  those  activities  normally 
associated  with  the  efficient  functioning  of  all 
organizations.  Categories  within  this  function  include: 

•  Financial  Management 

•  Personnel  Management 

•  Resource  Management 

•  Time  Management 

System  Engineering  Management  is  the  practical  application  of 
scientific  and  engineering  methods  and  techniques  to: 

•  Transform  mission  need  and  operational  requirement 

statements  into  a  description  of  a  system 

configuration  which  best  provides  the  required 
capabilities  in  accordance  with  the  specified 
effectiveness  metrics. 

•  Coordinate  the  system  design  definition  and  detail 

design  process. 

•  Integrate  all  technical  considerations  and  assure 

compatibility  of  all  physical,  functional,  and 
technical  interfaces  to  facilitate  an  optimized 
total  system  design. 

•  Integrate  the  activities  of  all  the  applicable 

engineering  disciplines  and  specialties  into  a 
unified  engineering  team  approach  to  facilitate  an 
effective  and  efficient  system  design  process. 

Integrated  Logistic  Support  (ILS)  Management  designs  and 

develops  the  logistic  support  system  concurrent  with  the 

objective  system  development  to  ensure  that: 

•  The  supportability  considerations  influence  the 
system  design  process. 

•  ILS  impacts  are  addressed  during  the  system  trade 
studies  in  the  early  project  phases. 

•  ILS  options  and  tradeoffs  can  be  considered  before 
the  system  design  is  frozen  or  the  product  baseline 
are  established. 

•  The  ultimate  logistic  system  is  an  integrated 
system  with  congruence  among  all  logistic  elements. 
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Production  Management  includes  those  activities  required  to 
produce  the  total  system  effectively  and  efficiently.  Tasks 
in  this  are  include: 

•  Production  planning  and  support. 

•  Production  and  industrial  engineering  for 

process,  facilities,  and  plant  equipment  design, 
development,  and  integration. 

•  Production  inventory  management  and  control. 
Configuration  Management  is  the  project  office  function 
responsible  for  documenting  the  functional  and  physical 
characteristics  of  the  objective  system  configuration.  Tasks 
encompassed  within  this  function  include: 

•  Technical  definition  and  documentation  of  the 
system  functional  and  physical  characteristics. 

•  Technical  and  administrative  control  of  all  changes 
to  the  systems  characteristics  and  attributes. 

•  Recording  and  reporting  change  processing  and 
implementation  status. 

Risk  Management  is  the  function  designed  to  force  organized 
purposeful  thought  to  the  subject  of  eliminating,  minimizing, 
or  containing  the  effects  of  undesirable  occurrences.  Risk 
management  is  a  continual  process  throughout  the  acquisition 
cycle.  Typical  risks  associated  with  the  acquisition  of  a 
major  weapon  system  include: 

•  Technical  Risk 

•  Programmatic  Risk 

•  Supportability  Risk 

•  Schedule  Risk 

•  Cost  Risk 

The  project  manager  attempts  to  eliminate  or  minimize  the 
occurrence  of  risk  by: 

•  Identifying  and  planning  for  potential  risk. 

•  Assessing  and  developing  Courses  of  Action  (COA)  to 

address  risk. 
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•  Analyzing  and  comparing  each  COA,  then  selecting 
the  best  suited  to  mitigate  risk. 

•  Implementing  the  selected  COA  to  avoid,  control, 
assume,  or  transfer  risk. 


E.  THE  CLOSE  COMBAT  ANTI -ARMOR  WEAPON  SYSTEMS  AND  THE  AIR- 
TO-GROUND  MISSILE  SYSTEMS  PROJECT  OFFICES 

Within  the  Army  acquisition  structure,  project  management 
offices  are  grouped  into  programs.  Each  program  or  Program 
Executive  Office  (PEO)  reports  directly  to  the  Assistant 
Secretary  of  the  Army  for  Research,  Development,  and 
Acquisition.  Currently  there  are  ten  such  programs,  each 
encompassing  the  development  of  major  systems  that  represent 
a  unique  capability  on  the  modern  battlefield  (i.e.  missiles, 
communication,  etc.) 


Figure  5:  The  Location  of  the  CCAWS  and  AGMS  Project  Offices 
within  the  Army  Acquisition  System  Structure. 
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The  CCAWS  and  AGMS  Project  Management  Offices  are  assigned  to 
the  PEO,  Tactical  Missiles,  located  at  Redstone  Arsenal, 
Alabama  (Figure  5) . 

1.  Air-to-Ground  Missile  Systems  (AGMS) 

AGMS  is  responsible  for  the  planning,  direction,  and 
control  of  tasks  involved  in  the  development  and  upgrade  of 
the  Hellfire,  Hellfire  II,  and  Longbow  Missile  Systems.  This 
includes  all  phases  of  development,  procurement,  production, 
distribution,  and  logistics  support  for  the  purpose  of 
maintaining  balanced  programs  to  accomplish  the  stated 
objectives  of  the  Army  Acquisition  Executive.  The  AGMS 
organizational  structure  is  depicted  in  Figure  6. 


Figure  6:  AGMS  Organizational  Diagram. 
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The  AGMS  project  office  also  exercises  operational  control 
over  personnel  assigned  to  U.S.  Army  Missile  Command  (MICOM) 
functional  organizations  collocated  with  the  project  office  to 
provide  support  in  the  areas  of  logistics,  product  assurance, 
engineering,  test,  and  configuration  management. 

2.  Close  Combat  Anti -armor  Weapon  Systems  (CCAWS) 

CCAWS  is  responsible  for  the  development  and  upgrade  of 
the  Tube  launched.  Optically  tracked.  Wire-guided  Weapon 
System  (TOW) ,  to  include  the  Improved  Target  Acquisition 
System  (ITAS),  the  Improved  Bradley  Acquisition  System  (IBAS), 
and  all  TOW  missile  configurations.  CCAWS,  as  with  AGMS, 
exercises  operational  control  over  personnel  assigned  to  MICOM 
functional  organizations  collocated  with  the  project  office. 
The  CCAWS  organizational  structure  is  depicted  in  Figure  7. 


Figure  7:  CCAWS  Organizational  Diagram. 
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IV.  THE  INSTRUMENTS 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  present  and  evaluate 
the  assessment  instruments  addressed  in  this  study  and  their 
supporting  models.  The  chapter  begins  with  a  brief  discussion 
of  the  theory  and  conceptual  framework  underlying  the 
development  and  implementation  of  organizational  assessments. 
It  then  presents  a  conceptual  model  developed  by  the  Harvard 
School  of  Business,  to  analyze  the  effectiveness  and 
interaction  of  organizational  components.  The  chapter 
concludes  with  the  description  and  evaluation  of  two 
assessment  models  and  their  assessment  instruments  used  to 
measure  the  program  health  and  customer  satisfaction  in  a 
project  management  office.  The  first  of  these  models  and 
assessment  instruments  were  used  by  the  CCAWS  and  AGMS  project 
offices  to  conduct  assessments  of  their  respective 
organizations  in  July  1994.  The  second  model  and  assessment 
instrument  were  selected  during  the  literature  review  as 
viable  alternatives  to  those  used  by  CCAWS  and  AGMS  in  July 
1994. 

B.  THEORY  AND  GENERAL  FRAMEWORK 

The  measurement  of  an  organization' s  productivity  and 
customer  satisfaction  is  a  crucial  managerial  task,  both  in 
the  public  and  private  sectors,  given  today's  environment  of 
declining  resources.  In  order  to  effectively  manage  their 
scarce  resources,  managers  must  develop  appropriate 
instruments  to  assess  and  improve  the  efficiency  of  their 
organization.  The  theory  underlying  the  development  and 
employment  of  such  measurement  instruments  is  termed  Total 
Quality  Management. 
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1 .  Total  Quality  Management 

a.  Philosophy 

Total  Quality  Management  (TQM)  is  a  philosophy,  as 
well  as  a  set  of  guiding  principles,  that  represents  the 
foundation  of  continuous  improvement  in  organizations.  It 
applies  both  qualitative  and  quantitative  methods  to  improve 
the  materials  and  services  provided  by  an  organization,  and 
seeks  to  meet  the  customer's  needs,  both  now  and  in  the 
future.  The  application  of  TQM  integrates  fundamental 
management  techniques,  existing  improvement  efforts,  and 
technical  tools  in  an  effort  to  continually  improve  all 
organizational  processes. 

TQM  addresses  the  quality  of  management  as  well  as 
the  management  of  quality.  It  involves  everyone  in  the 
organization  in  a  long  term  endeavor  to  develop  processes  that 
are  customer  oriented,  flexible,  responsive,  and  constantly 
improved  in  quality.  Since  quality  includes  any  factor  of  a 
product  or  service  of  value  to  a  customer,  TQM  seeks  to  create 
an  environment  to  sustain  a  culture  committed  to  the 
continuous  quality  improvement  of  these  factors.  TQM  is  also 
a  means  for  improving  personal  effectiveness  and  performance, 
and  for  aligning  and  focusing  all  efforts  throughout  the 
organization.  (Mansi,  1989) 

b.  Quality  Measurement 

As  TQM  principles  are  implemented  within  an 
organization,  the  measurements  of  quality  and  customer 
satisfaction  become  crucial  elements  in  achieving  improved 
productivity.  Quality  results  from  the  successful  integration 
of  all  an  organization's  functions  to  deliver  first  rate 
products  and  services.  Customer  satisfaction  is  the  result  of 
continuous  quality  control  of  the  delivered  products  and 
services.  Since  all  organizational  functions  play  a  part  in 
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quality  performance,  each  critical  function  within  the 
organization  should  be  measured  to  determine  the  level  of 
quality  at  which  the  organization  is  functioning. 

Civilian  organizations  measure  quality  performance 
using  several  methods.  One  method  use  is  Statistical  Process 
Control  (SPC) .  Organizations  using  this  method  collect 
performance  data,  such  as  rework  or  defect  rates,  that  focus 
on  the  production  function  of  the  organization.  Although  this 
is  a  valid  method,  it  is  narrow  in  focus  and  does  not  reflect 
organization-wide  quality  management  efforts.  Another  form  of 
measurement  that  is  used,  and  one  that  does  more  readily 
identify  the  impact  of  organization-wide  quality  management  is 
measurement  based  on  actions.  That  is,  measurement  techniques 
that  record  customer  feedback  on  products,  measure  the  number 
of  customer  complaints,  or  measure  other  such  external  data. 
This  method  is  also  narrowly  focused  and  fails  to  address 
quality  issues  internal  to  the  organization.  Still,  other 
organizations  measure  financial  performance  to  gage  quality 
within  their  organization.  This  form  of  quality  measurement 
does  not  take  into  account  customer  satisfaction,  nor  does  it 
fully  consider  organizational  management  techniques  and  other 
critical  factors.  (Saraph,  Benson,  and  Schroeder,  1989) 

Another  quality  measurement  instrument  used  by 
civilian  organizations  is  the  organizational  self-assessment. 
This  type  of  measurement  is  considered  by  many  to  be  the  most 
comprehensive  technique  currently  used  in  organizations.  The 
use  of  self-assessment  instruments  provides  an  organization's 
management  with  increased  flexibility,  allowing  management  to 
modify  the  assessment  instrument,  as  needed,  to  reflect  the 
organization's  changing  environment,  values,  and  procedures. 
The  most  widely  used  and  popular  self-assessment  instrument  is 
the  Malcolm  Baldrige  National  Quality  Award.  The  Baldrige 
award  was  created  to  recognize  the  achievements  of  U.S. 
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companies  that  excel  in  the  implementation  of  successful 
quality  strategies.  It  is  a  highly  respected  accolade  and  a 
useful  diagnostic  tool  that  measures  the  organization's 
overall  quality  management  and  performance.  The  Baldrige 
Award  incorporates  a  team  based,  consensus  approach  by  the 
organization' s  management  to  measure  of  a  set  of  seven  quality 
criteria,  which  are  in  turn  used  to  improve  the  organization's 
performance.  (ASQC,  1995) 

Just  as  the  use  of  TQM  concepts  have  become  the 
vehicle  for  improving  productivity  and  customer  satisfaction 
in  all  sectors  of  private  industry,  so  have  they  become  the 
impetus  for  improvement  in  Federal  organizations.  Subsequent 
to  the  onset  of  "quality  consciousness"  within  the  private 
sector,  President  Clinton  encouraged  Federal  managers  at  all 
levels  to  manage  quality  improvement  in  accordance  with 
criteria  established  by  the  Presidential  Award  for  Quality. 
These  criteria,  extracted  and  modified  from  the  Baldrige 
Award,  guide  Federal  agencies  in  striving  toward  increased 
quality  management  by  improving  operational  performance  and 
customer  satisfaction.  The  National  Performance  Review  Report 
of  September  1993  reinforced  this  viewpoint  suggesting  that 
all  Federal  agencies  adopt  these  criteria  and  conduct  regular 
assessments  using  the  Presidential  Award' s  implementing 
instrument,  the  Self  Assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction,  as  a  means  of 
identifying  areas  needing  improvement.  (Federal  Quality 
Institute,  1993) 

Recalling  the  principal  focus  of  this  study,  which 
is  to  develop  an  assessment  instrument  to  effectively  measure 
the  program  health  and  customer  satisfaction  of  a  project 
management  office,  one  must  begin  by  analyzing  the  assessment 
instrument  currently  being  used,  as  well  as  viable 
alternatives  to  the  same.  A  study  of  these  assessment 
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instruments,  however,  would  be  fruitless  without  first 
considering  the  basis  for  the  instrumentation  design. 
Therefore,  this  study  will  first  address  the  assessment  models 
supporting  each  of  the  instruments  to  be  considered,  followed 
by  a  discussion  of  the  assessment  instruments  themselves.  In 
order  to  effectively  analyze  and  compare  the  models  identified 
in  this  study,  the  Harvard  Business  School  model  will  be  used. 
The  Harvard  Business  School  model  was  chosen  as  the  conceptual 
basis  for  comparison  because  it  is  the  most  comprehensive  tool 
available  for  use  in  analyzing  the  interrelationships  between 
various  critical  organizational  activities. 

2.  The  Harvard  Business  School  Model  (modified) 

The  Harvard  Business  School  model  was  developed  as  an 
analysis  tool  to  educate  its  students  in  the  factors  involved 
in  strategic  planning  and  management.  The  model  has 
subsequently  been  adopted  and  used  by  Dr.  Nancy  C.  Roberts  for 
instruction  at  the  Naval  Postgraduate  School,  Monterey, 
California.  In  addition  to  teaching  this  model.  Dr.  Roberts 
has  successfully  applied  the  model,  as  a  consultant,  while 
analyzing  the  strategic  planning  factors  of  numerous  private 
sector,  state  government,  and  Federal  Government 
organizations.  Based  on  her  exposure  to  this  model,  through 
research  and  practical  application,  Dr.  Roberts  has  made 
several  modifications  to  the  original  Harvard  model.  The 
Harvard  model,  as  modified  by  Dr.  Roberts  (Figure  8),  is  used 
by  this  researcher  as  the  basis  to  evaluate  the  models  that 
support  the  assessment  instruments  addressed  in  this  study. 
Each  model  is  evaluated  against  the  Harvard  model  to  determine 
the  comprehensiveness  in  which  it  addresses  the  critical 
organizational  design  factors  that  determine  the  success  or 
failure  of  an  organization. 
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Strategic  Inputs  Strategic  Evaluation 


Figure  8:  The  Harvard  Business  School  Model  (modified). 


The  Harvard  Business  School  model  (modified)  focuses  its 
analysis  in  four  general  areas: 

•  Strategic  Inputs 

•  Direction  Setting 

•  Strategic  Implementation 

•  Strategic  Evaluation 

The  interrelationships  between  these  areas  and  their  internal 
elements  are  indicated  with  solid  arrows.  Dashed  arrows 
represent  feedback  from  one  area  to  another. 

The  first  area.  Strategic  Inputs,  consists  of  two 
categories:  Environmental  Factors,  and  Key  Success  Factors. 
Environmental  Factors  -  are  defined  as  elements  that  affect  an 
organization  of  which  it  has  no  direct  control.  Examples  of 
environmental  factors  include:  The  status  of  the  economy,  the 
level  of  competition  in  the  industry,  and  social  and  political 
change . 
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Key  Success  Factors  -  are  critical  elements  required  for  an 
organization  to  succeed  in  a  particular  endeavor,  given  the 
aforementioned  environmental  factors.  Examples  of  key  success 
factors  are:  Responsiveness,  Marketing,  and  the  Delivery  of 
Goods  or  Services. 

The  second  area  addressed  by  the  Harvard  model  is 
Direction  Setting.  The  components  within  this  area 
collectively  represent  top  level  management's  plan  for  the 
organization  to  accomplish  its  key  success  factors,  given  the 
current  environmental  factors.  The  elements  of  direction 
setting  include: 

•  The  Mission 

•  The  Vision 

•  Strategic  Issues 

•  Strategies 

•  Goals  and  Objectives 

The  Mission  is  a  general  statement  describing  the 

organization's  overall  purpose  or  reason  for  being. 

The  Vision  -  of  an  organization  conceptualizes  the  desired 
endstate  of  the  organization  at  mission  accomplishment. 
Strategic  Issues  -  are  items  of  critical  focus,  determined  by 
senior  management,  that  relate  to  the  successful 
accomplishment  of  the  organization' s  stated  mission  and  key 
success  factors. 

Strategies  -  are  management's  overarching  plans  formulated  to 
address  the  identified  strategic  issues. 

Goals  -  are  defined  as  interim  success  points  established  by 
management.  Goals  are  used  to  measure  the  organization's 
progress  towards  accomplishing  their  defined  strategies. 
Objectives  -  are  the  basic  building  blocks  or  steps  that 
comprise  the  organization's  established  goals. 
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The  third  area  addressed  by  the  Harvard  model  is 

Strategic  Implementation.  This  area  is  focused  on  the 
identification  and  assessment  of  the  organization's  design 
factors,  as  well  as  their  interaction  with  one  another. 
Organizational  design  factors,  according  to  the  Harvard  model 
include : 

•  Task  Reguirements 

•  Technology 

•  People 

•  Organization  Structure 

•  Process/Systems 

Task  Requirements  -  are  defined  as  the  basic  activities  an 
organization  must  perform  effectively  to  accomplish  its 
mission  and  achieve  its  key  success  factors,  given  the 
environmental  factors. 

Technology  -  is  defined  as  the  basic  methods  or  techniques 
used  within  the  organization. 

People  -  refers  to  all  human  resources  available  to  the 
organization. 

Organization  Structure  -  focuses  on  the  basic  grouping  of  the 
organization's  activities  and  people.  Structure  is  also 
concerned  with  the  devices  used  by  the  organization  to 
integrate  these  activities  and  people. 

Processes/Systems  -  is  focused  on  the  various  processes  and 
systems  in  place  within  the  organization  to  guide  the 
remaining  organizational  design  factors  towards  the 
achievement  of  the  key  success  factors  and  mission 
accomplishment.  Critical  processes/systems  identified  by  the 
Harvard  model  include: 

•  Measurement  and  Control 

•  Training  and  Development 

•  Rewards 
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•  Recruitment  and  Selection 

The  final  area  evaluated  by  the  Harvard  model  is 

Strategic  Evaluation.  This  area  is  comprised  of: 

•  Culture  and  Management  Style 

•  Outputs 

•  Outcomes 

•  Problems 

Culture  and  Management  Style  -  is  the  pattern  of  past 
behavior,  activity,  and  managerial  effectiveness  which  may 
have  an  affect  on  the  current  or  future  functioning  of  the 
organization. 

Outputs  -  are  clearly  identifiable  and  measurable  products  or 
services  provided  by  an  organization. 

Outcomes  -  are  the  consequences  or  results  affecting  the 

organization  or  its  environment  related  to  the  organization's 
outputs . 

Problems  -  are  defined  as  critical  flaws  or  malfunctions 

associated  with  one  dr  more  of  the  previously  defined 
categories . 

C .  INSTRUMENT  EVALUATION 

1.  The  Presidential  Award  for  Quality  and  the  Unit  Self- 
Assessment  Guide  for  Organizational  Performance  and 
Customer  Satisfaction 

a.  The  Presidential  Award  for  Quality 

The  Presidential  Award  for  Quality  was  instituted  in 
1988.  The  award  was  created  to  recognize  organizations  that 
have  implemented  quality  management  in  an  exemplary  manner, 
resulting  in  high  quality  products  and  services,  the  effective 
use  of  taxpayer  dollars,  and  to  promote  quality  management 
awareness  and  implementation  throughout  the  Federal 
Government.  (Federal  Quality  Institute,  1995,  p.  1) 
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The  Presidential  Award  model  addresses  seven  quality 
criteria,  common  to  most  Federal  organizations.  These 
criteria  (Figure  9)  were  extracted  and  modified  from  the 
Baldrige  model  to  guide  Federal  agencies  in  striving  toward 
increased  quality  management,  improved  operational 
performance,  and  customer  satisfaction.  As  with  the  Harvard 
model,  the  relationships  between  each  of  the  areas  are 
indicated  with  solid  arrows. 


Figure  9:  The  Presidential  Award  Model. 

According  to  the  Presidential  Award  criteria: 
Leadership  -  is  defined  as  the  executives'  personal  commitment 
and  involvement  in  creating  and  sustaining  an  organizational 
vision  and  customer  focus  orientation,  as  well  as  clear  and 
visible  quality  values.  The  evaluation  of  this  criterion  is 
concerned  with  how  the  vision,  values  and  customer  focus 
orientation  are  integrated  into  the  management  system,  labor 
relations,  and  external  partnerships. 
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Management  of  Process  Quality  -  addresses  the  systematic 
processes  used  by  the  organization  for  continuous  improvement 
of  quality  and  operational  performance,  the  design  and 
management  of  process  quality  for  all  work  units,  management 
of  internal  customer-supplier  relationships,  supplier  and 
intermediary  quality,  and  quality  assessment. 

Human  Resource  Development  and  Management  -  examines  how  the 
entire  workforce  is  enabled  to  develop  its  full  potential,  and 
to  pursue  quality  and  operational  performance  improvement 
goals.  It  also  addresses  efforts  to  build  and  maintain  an 
environment  for  workforce  excellence. 

Strategic  Quality  Planning  -  focuses  on  the  organization's 
planning  process,  both  short  and  long-term,  and  how  key 
quality  requirements  are  integrated  into  the  overall  planning 
process . 

Information  and  Analysis  -  examines  the  scope,  management,  and 
use  of  data,  information  and  measures,  and  how  they  are  used 
to  drive  quality  and  operational  performance  improvement.  It 
also  examines  the  adequacy  of  the  organization's  data, 
information,  and  analysis  system  to  support  the  improvement  of 
customer  satisfaction,  products,  services,  and  processes. 
Customer  Focus  and  Satisfaction  -  encompasses  the 
organization's  knowledge  of  external  customer  requirements, 
the  techniques  used  to  establish  and  maintain  customer 
relationships,  and  the  methods  used  to  determine  customer 
satisfaction. 

Quality  and  Operational  Results  -  considers  the  organization' s 
trends  and  quality  levels  for  products  and  services, 
operational  performance,  business  processes  and  support 
services,  supplier  and  intermediary  quality,  and 
comparison/benchmark  data. 
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Evaluating  the  above  criteria  against  the  Harvard 
Business  School  model,  we  find  that  the  Presidential  Award 
Criteria,  although  appropriate  to  assess  the  quality 
improvement  of  an  organization,  does  very  little  in  the  way  of 
addressing  the  interaction  and  effectiveness  of  its 
organizational  components.  A  comparison  of  the  two  models 
(Figure  10)  reveals  that  the  factors  in  the  Harvard  model  that 
are  also  addressed  by  the  Presidential  model  are  limited  to: 

•  Mission,  Vision,  Strategic  Issues,  Strategies,  and 
Goals/Objectives 

•  Process/Systems 

•  People 

•  Outputs 

•  Outcomes 
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Figure  10:  A  Comparison  of  the  Presidential  Award  Model  and 
the  Harvard  Business  School  Model. 
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In  the  above  diagram,  factors  that  are  similar  in  both  models 
are  connected  with  solid  arrows.  Factors  in  the  Harvard  model 
that  are  not  addressed  by  the  Presidential  model  are 
represented  with  darkened  boxes. 

Even  though  the  factors  listed  above  are  addressed  by 
both  the  Harvard  model  and  the  Presidential  model,  the  clear 
focus  of  the  Presidential  Award  criteria  is  to  measure  the 
quality  improvement  efforts  in  each  of  these  areas,  rather 
than  their  individual  effectiveness  and  interaction  with  one 
another.  Therefore,  the  Presidential  Award  model  is  not  the 
most  appropriate  model  for  use  in  measuring  both  the  program 
health  and  customer  satisfaction  of  an  organization. 

b.  The  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction 

To  assist  Federal  organizations  in  conducting  a  unit 
self-assessment  prior  to  submitting  an  application  for  the 
Presidential  Award,  the  Unit  Self-assessment  Guide  for 
Organizational  Performance  and  Customer  Satisfaction,  based  on 
the  award's  criteria,  was  developed  and  made  available  to  all 
Federal  organizations.  The  self-assessment  consists  of  66 
individual  items,  grouped  in  categories  aligned  with  the 
Presidential  Award  criteria.  This  instrument,  as  with  the 
Baldrige  Award,  incorporates  a  team  based,  consensus 
management  approach  to  evaluate  each  item. 

The  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction  was  designed  to  be 
universally  applied  to  all  Federal  organizations.  However, 
after  conducting  a  self-assessment  using  the  Self-assessment 
Guide  for  Organizational  Performance  and  Customer  Satisfaction 
in  July  1994,  the  management  teams  and  assessment  participants 
from  the  CCAWS  and  AGMS  project  management  offices  identified 
several  shortcomings  with  the  instrument  and  its  use  in 
measuring  the  critical  organizational  factors  within  a  project 
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management  office.  The  shortcomings  identified  were: 

•  The  assessment  was  lengthy  and  time  consuming. 

•  The  assessment  items  were  wordy  and  ambiguous. 

•  Many  assessment  items  did  not  apply  to  a  project 
management  office. 

•  The  assessment  did  not  address  several  critical 
functional  areas  within  a  project  management 
office . 

•  The  assessment  did  not  provide  a  vehicle  to  collect 
qualitative  feedback. 

Given  the  above  shortcomings  and  the  fact  that  this  assessment 
was  constructed  using  the  Presidential  Award  criteria  as  a 
basis,  this  researcher  concludes  that  the  Unit  Self-assessment 
Guide  for  Organizational  Performance  and  Customer 
Satisfaction,  as  with  the  Presidential  Award  Criteria,  is 
inappropriate  for  use  in  measuring  the  program  health  and 
customer  satisfaction  of  a  project  management  office. 

2.  The  Strategic  Leadership  Planning  Model  and  the 
USAREUR  Organizational  Diagnostic  Survey 

a.  The  Strategic  Leadership  Planning  Model 

Based  on  the  shortcomings  identified  with  the 
Presidential  Award  for  Quality  and  the  Self-assessment  Guide 
for  Organizational  Performance  and  Customer  Satisfaction,  a 
literature  review  was  conducted  to  locate  existing  models  and 
unit  self-assessment  instruments,  within  the  public  and 
private  sectors  that  meet  the  project  managers'  requirements 
as  potential  alternatives  to  the  above  model  and  assessment 
instrument.  Of  the  available  models  and  assessment 

instruments,  the  Strategic  Leadership  Planning  model  and  the 
United  States  Army,  Europe  (USAREUR)  Organizational  Diagnostic 
Survey,  based  on  the  Strategic  Planning  model,  were  selected 
as  the  model  and  instrument  whose  framework  most  accurately 
reflect  the  critical  factors,  identified  by  the  CCAWS  Project 
Manager  and  the  AGMS  Deputy  Project  Manager,  that  determine 


the  program  health  and  customer  satisfaction  in  a  project 
office . 

The  program  health  of  a  project  management  office  is 
defined  by  these  managers  as: 

The  efficient  functioning  of  an  organization 
that  results  in  the  successful  fielding  and  support 
of  material  to  its  designated  users,  where  the 
organizational  climate  is  based  on  teamwork  and  is 
one  in  which  subordinates  are  empowered  and 
resourced  to  accomplish  their  assigned  tasks; 
decisions  are  made  at  the  lowest  appropriate  level 
within  the  organization;  personnel  enjoy  their 
working  environment;  and  workers  have  confidence  in 
the  products  and  services  they  deliver. (Interviews, 

1995) 


Customer  satisfaction,  for  the  purpose  of  this 
study,  is  defined  as: 

An  organization's  demonstrated  success  in 
meeting  the  requirements  of  the  recipients  to  which 
services  are  performed  or  products  are  delivered  by 
the  organization.  (Hunt,  1993) 

The  Strategic  Leadership  Planning  model  (Figure  11) 
addresses  four  general  factor  categories.  (DA,  CGSC,  Strategic 
Planning,  1994)  These  categories  are: 

•  Inputs 

•  Organizational  Strategy 

•  Organizational  Design  Factors 

•  Outputs 

Included  in  the  Inputs  category  are  factors  external  to 
and  within  an  organization  that  are  present  and  necessary  to 
be  considered  at  the  outset  of  the  strategic  planning  process. 
These  Factors,  according  to  the  Strategic  Leadership  Planning 
model,  are: 


Purpose 

Environment 


Culture 


Expectations 
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Figure  11:  The  Strategic  Leadership  Planning  Model. 


•  Demands 

•  Opportunities 

Purpose  -  is  defined  as  a  broad,  general  definition  of  the 
organization's  reason  for  being  in  existence. 

Environment  -  as  in  the  Harvard  model,  encompasses  those 
elements  affecting  the  organization  that  are  beyond  the  direct 
control  of  the  organization's  management. 

Culture  -  is  the  pattern  of  past  behavior,  activity,  and 
effectiveness  which  may  have  an  affect  on  the  current  or 
future  functioning  of  the  organization. 

Expectations  -  are  standards  or  achievements,  internal  and 
external  to  the  organization,  considered  reasonable,  due,  or 
necessary  to  be  obtained. 

Demands  -  are  defined  as  requirements  of  work  or  of  the 
expenditure  of  resources. 
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Opportunities  -  are  favorable  junctures  in  circumstances  that 
produce  a  good  chance  for  advancement  or  progress . 

The  organizational  strategies  category  encompasses 
factors  that  comprise  top  management's  overarching  plan  for 
operating  the  organization.  These  factors  include: 

•  Values 

•  Mission 

•  Goals 

•  Objectives 

Values  -  are  defined  as  those  core  beliefs  held  by  management 
that  are  assimilated  into  and  guide  the  formulation  of  the 
organization's  mission,  goals,  and  objectives. 

The  mission,  goals,  and  objectives  factor  definitions  are 
similar  to  the  factors  of  the  same  names  previously  defined  in 
the  description  of  the  Harvard  Business  School  model. 

The  next  category  addressed  by  the  Strategic  Leadership 
Planning  model  is  Organizational  Design  Factors.  Factors  in 
this  category  include: 

•  Technology 

•  Rewards 

•  Tasks 

•  Leadership 

•  Processes 

•  Structure 

•  People 

The  technology,  tasks,  processes,  structure,  and  people 

factor  definitions  do  not  significantly  deviate  from  those 
factors  of  the  same  names  previously  defined  in  the 
description  of  the  Harvard  Business  School  model. 

Rewards  -  are  defined  as  the  intrinsic  and  extrinsic 
motivators  of  people. 
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Leadership  -  is  defined  as  the  art  of  direct  and  indirect 
influence  and  the  skill  of  creating  the  conditions  for 
sustained  organizational  success  to  achieve  a  desired  result. 
(DA,  FM  22-103,  1987,  p.  3) 

The  final  category  considered  by  the  Strategic  Leadership 
Planning  model  is  Outputs.  As  with  several  factors  in  the 
above  categories,  the  Strategic  Planning  model's  definition  of 
outputs  is  similar  to  that  defined  in  the  Harvard  Business 
School  model.  The  key  difference  being  that  outputs, 
according  to  the  Strategic  Leadership  Planning  model,  are 
grouped  into  three  distinct  categories:  Organizational, 

Group,  and  Individual. 

Organizational  Outputs  -  are  the  formal  products  or  services 
produced  by  the  organization. 

Group  Outputs  -  are  the  products  of  various  groups  or 
subsections  of  the  organization,  intra-organizational  products 
or  services. 

Individual  Outputs  -  are  those  products  produced  as  the  result 
of  the  efforts  of  a  single  individual. 

Comparing  the  Strategic  Leadership  Planning  model  to  the 
Harvard  Business  School  model,  it  is  clear  that  the  Strategic 
Leadership  Planning  model  more  accurately  reflects  the  factors 
addressed  by  the  Harvard  model  as  critical  to  determine  the 
health  of  an  organization.  This  comparison  (Figure  12)  shows 
a  substantive  increase  in  the  areas  covered  by  both  models. 
Areas  addressed  in  the  Harvard  model  and  covered  in  the 
Strategic  Planning  model  include  the  areas  addressed  by  both 
the  Harvard  model  and  the  Presidential  model,  plus: 

•  Environment 

•  Task  Requirements 

•  Technology 
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Figure  12:  A  Comparison  of  the  Strategic  Leadership 
Planning  Model  and  the  Harvard  Business  School  Model, 


•  Structure 

•  Culture  and  Management  Style 

As  with  the  previous  comparison,  factors  that  are  similar  in 
both  models  are  connected  with  solid  arrows.  Factors  in  the 
Harvard  model  that  are  not  addressed  by  the  Strategic 
Leadership  Planning  model  are  represented  with  darkened  boxes. 
There  is  one  category,  outcomes,  that  is  covered  by  both  the 
Harvard  model  and  the  Presidential  model  that  is  not  addressed 
by  the  Strategic  Leadership  Planning  model. 

Although  the  Presidential  model  addressed  some  of  the 
same  general  categories  as  the  Harvard  model,  as  discussed 
earlier,  the  definitions  and  intent  of  these  categories  were 
fundamentally  different.  This  is  not  the  case  with  the 
Strategic  Leadership  Planning  model.  As  noted  above,  many  of 
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the  factor  definitions  used  in  the  Strategic  Leadership 
Planning  model  are  similar  in  nature  and  intent  to  those 
defined  by  the  Harvard  model. 

b.  The  USAREUR  Organizational  Diagnostic  Survey 
The  USAREUR  Organizational  Diagnostic  Survey 
(Appendix  B)  was  developed  in  1982  by  the  USAREUR 
Organizational  Effectiveness  Office  as  an  implementing 
instrument  for  the  Strategic  Leadership  Planning  model.  The 
purpose  of  the  survey  was  to  solicit  feedback  concerning  an 
organizations  values,  organizational  design  factors,  and 
outputs  from  each  member  within  the  organization.  The 
original  USAREUR  Diagnostic  Survey  consisted  of  66  individual 
items,  grouped  into  eight  categories  aligned  with  the  seven 
organizational  design  factors  and  the  organizational  strategy 
category  identified  by  the  Strategic  Planning  model.  The 
survey  was  modified  by  this  researcher  and  administered  to 
selected  members  of  the  CCAWS  and  AGMS  organizations  to 
determine  if  the  survey  and  its  supporting  model  address  the 
shortcomings  identified  with  the  Presidential  Award  model  and 
the  Self-assessment  Guide  for  Organizational  Performance  and 
Customer  Satisfaction  and  are  effective  tools  to  measure  the 
program  health  and  customer  satisfaction  of  a  project 
management  office.  The  modified  survey  (Appendix  A) 
administered  during  the  prototype  assessment  testing  consists 
of  74  items,  grouped  as  above. 
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V.  INSTRUMENT  AUTOMATION 


A.  INTRODUCTION 

The  sponsoring  organizations  of  this  thesis  requested 
that  the  assessment  instrument  recommended  by  this  study 
incorporate  a  automated  approach  to  the  assessment  process. 
This  chapter  describes  the  candidate  software  programs 
selected  as  potential  solutions  to  this  requirement.  The 
chapter  first  discusses  this  researcher's  evaluation  of  the 
Continuous  Improvement  (Cl)  Toolkit  to  meet  the  sponsors' 
specified  requirements.  The  chapter  then  presents  an 
independently  developed  software  program,  designed 
specifically  for  the  purpose  of  administering  the  assessment 
instrument  recommended  by  this  study. 

B.  THE  CONTINUOUS  IMPROVEMENT  TOOLKIT  FOR  WINDOWS 

The  Continuous  Improvement  Toolkit  for  Windows  was 
developed  by  Systems  Improvement,  Incorporated  in  1984.  The 
program  was  designed  as  a  medium  to  determine  the  status  of  an 
organization,  measured  against  the  management's  goals  and 
objectives,  using  an  appropriate  quality  model.  It  was 
further  designed  to  assist  management  to  implement  a  plan  for 
the  correction  of  identified  shortcomings;  define,  measure, 
and  track  their  root  causes;  and  benchmark  the  organization's 
progress  against  world  class  companies.  (Interviews,  1995) 

After  interviewing  the  developers  of  the  Cl  Toolkit  and 
completing  the  workshop  provided  by  Systems  Improvement,  this 
researcher  evaluated  the  Cl  Toolkit' s  appropriateness  for  use 
in  administering  the  USAREUR  Organizational  Diagnostic  Survey 
and  the  assessment  recommended  by  this  study.  The  evaluation 
concluded  that  the  Cl  Toolkit,  as  designed,  does  not  meet  the 
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sponsors'  specific  requirements  and  is  not  suitable  to 
administer  the  above  assessments  for  the  following  reasons: 

•  The  Cl  Toolkit  was  not  designed  to  collect  feedback 
from  100  percent  of  an  organization.  It  is 
primarily  developed  for  use  by  organizations  which 
intend  to  prepare  an  application  for  a  quality 
achievement  award,  such  as  the  Baldrige.  It 
therefore  incorporates  a  consensus-based  approach 
to  data  collection.  Data  collected  by  this  process 
are  recorded  as  a  single  response. 

•  Data  are  stored  to  a  flat,  non-relational  database 
file . 

•  Data  recorded  by  the  Cl  Toolkit  are  stored  in  a 
unique  format  that  is  incompatible  with  the 
analysis  and  database  programs  currently  used  by 
the  sponsoring  organizations. 

•  Analyses  conducted  within  the  Cl  Toolkit  program 
are  fixed  and  limited  to  summary  statistics. 

•  The  Cl  toolkit  provides  a  limited  number  of 
answering  scales.  Once  chosen,  one  scale  must  be 
used  to  answer  all  items  in  the  assessment  or 
survey. 

•  The  display  screens  used  to  present  the  assessment 
items  are  copyrighted  and  cannot  be  tailored  to 
maximize  the  administration  of  the  recommended 
survey. 

•  The  Cl  Toolkit  is  not  a  cost  effective  solution. 
The  sponsors'  require  only  one  of  the  seven  major 
features  included  in  the  software  program. 


C.  PROTOTYPE  SOFTWARE  DEVELOPMENT 

Based  on  the  evaluation  of  the  Cl  Toolkit  for  Windows  and 
the  sponsors'  specific  automation  requirements,  this 
researcher  developed  an  independent  prototype  software  program 
specifically  designed  to  administer  the  assessment  recommended 
by  this  study.  The  prototype  consists  of  two  major  parts:  a 
network-based  user  interface  application  for  the  purpose  of 
data  collection,  and  a  data  storage,  analysis,  and 
presentation  application. 
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1 .  User  Interface 

The  user  interface  was  designed  using  Novell's  AppWare. 
AppWare,  version  1.2,  is  a  commercially  available  visual 
application  builder  developed  by  the  Novell  Corporation. 
AppWare  enables  users  to  develop  independent  software 
applications  using  Object  Oriented  Design  techniques.  The 
program  offers  the  user  a  selection  of  various  standard 
subjects,  objects,  and  operators  common  to  the  windows 
environment.  These  subjects,  objects,  and  operators  appear  to 
the  user  as  icons  (Figure  13) . 
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Figure  13:  AppWare  Objects  and  Operators. 


Each  icon  represents  a  unique  set  (lines)  of  code 
required  to  perform  a  selected  activity.  The  code,  itself,  is 
transparent  to  the  user,  thus  allowing  users  untrained  in 
writing  code  to  develop  complex  applications.  To  design  a 
program  the  user  simply  drafts  the  desired  program  hierarchy, 
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structure,  and  associated  logic,  places  the  necessary  objects 
and  operators  within  the  developed  structure,  assigns 
parameters  for  each  object  and  operator,  complies  the 
application,  then  runs  the  program. 

The  purpose  of  the  user  interface  is  to  provide  a  network 
based,  user  friendly  interface  for  data  collection  in  a 
windows  environment.  The  program  is  designed  to  accomplish 
the  following  tasks: 

•  Instruct  users  on  the  assessment  procedures. 

•  Administer  the  assessment. 

•  Collect  respondent  feedback  to  the  assessment 
items . 

•  Temporarily  store  the  collected  data. 

•  Export  data  to  a  common  data  format . 

•  Initiate  the  data  storage,  analysis,  and 
presentation  program. 

The  user  interface  is  designed  for  installation  onto  an 
organization's  Local  Area  Network  (LAN).  The  data  flow  using 
the  LAN  is  depicted  in  Figure  14. 


Figure  14:  User  Interface  Network  Diagram. 
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The  LAN  is  the  most  efficient  method  to  administer  an 
assessment  within  a  large  or  complex  organization  and  provides 
an  added  degree  of  flexibility  and  comfort  to  both  the 
facilitator  and  participants  of  the  assessment.  Data  are  input 
by  each  respondent  from  any  available  user  workstation 
connected  to  by  LAN  to  the  network  server.  These  data  are 
sent  to  the  network  server  and  stored  in  the  input  database. 
After  all  data  have  been  entered,  the  unit  facilitator,  from 
his/her  workstation  transfers  the  data  from  the  input  to  the 
analysis  package.  Data  and  analyses  stored  in  the  analysis 
package  can  be  viewed  by  the  unit  facilitator  or  the 
project/product  managers  from  their  respective  workstations. 

The  user  interface  presents  the  assessment  items  using 
the  standard  Windows  graphical  environment.  Respondents  are 
presented  a  series  of  screens,  each  containing  an  assessment 
item  and  an  appropriate  scale  to  respond  to  the  item.  (Figure 
15) 
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Figure  15:  Example  Assessment  Item  Screen. 
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The  respondent  simply  indicates  the  appropriate  response  to 
each  item  then  continues  to  the  next  item.  Once  entered,  each 
response  is  converted  to  a  numerical  value  and  stored  in  the 
input  database. 

2.  Data  Storage,  Analysis,  and  Presentation 

The  data  storage,  analysis,  and  presentation  application 
was  designed  using  Microsoft's  Assess  database  program.  The 
application  is  designed  to  accomplish  the  following  tasks 
using  parameters  defined  in  the  application  development: 

•  Import  data  from  the  user  interface  program 

•  Format  the  data 

•  Query  the  data 

•  Analyze  the  data 

•  Present  analysis  results 

•  Print  the  data  and  analyses 

•  Store  the  data  for  a  five  year  period 
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Figure  16:  Example  Analysis  Presentation. 


Average  Score 
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Once  imported  from  the  interface  database,  data  are 
automatically  formatted  and  sorted.  Queries  are 
simultaneously  constructed  based  on  the  parameters  stated 
during  the  application  development.  Calculations  used  to 
analyze  the  data  are  completed  as  part  of  the  querying 
process.  Analysis  results  are  presented  graphically  beginning 
with  a  macro  view  of  the  overall  assessment  (Figure  16)  . 
Successive  levels  of  detail  can  be  viewed  instantly  by 
indicating  which  category  or  demographic  feature  is  desired. 
This  process  is  facilitated  through  the  use  of  macros  included 
in  the  MS  Access  program  and  logic  statements  designed  during 
the  application  development. 
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VI.  ANALYSIS  AND  DISCUSSION 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  analyze  the  data  and 
feedback  accumulated  during  the  candidate  assessment  and 
prototype  software  testing,  and  to  discuss  issues  concerning 
the  applicability  and  effectiveness  of  the  USAREUR 
Organizational  Diagnostic  Survey  and  the  prototype  self- 
assessment  software.  The  chapter  begins  with  the  presentation 
of  summarized  data  and  qualitative  responses  from  the  surveys 
and  interviews  conducted  by  this  researcher.  These  data  and 
responses  are  then  used  to  analyze  the  appropriateness  of  the 
individual  items  contained  in  the  USAREUR  Organizational 
Diagnostic  Survey  (modified)  to  measure  the  program  health  and 
customer  satisfaction  of  a  project  management  office.  The 
chapter  concludes  with  a  discussion  of  issues  surfaced  during 
the  prototype  testing  concerning  the  design  and  procedures  of 
the  USAREUR  Organizational  Diagnostic  Survey  (modified)  and 
the  prototype  software  application. 

B.  DATA 

1 .  Survey  #1 

As  previously  addressed,  the  analysis  of  the  data 
collected  concerning  the  program  health  and  customer 
satisfaction  of  the  CCAWS  and  AGMS  project  offices  is  beyond 
the  scope  of  this  study  and  therefore  will  not  be  presented 
here.  These  data  were  collected  to  facilitate  the  candidate 
assessment  and  prototype  software  testing  and  to  provide 
realistic  data  for  use  in  the  development  of  the  data  storage, 
analysis,  and  presentation  application.  Data  recorded  during 
the  prototype  testing  will,  however,  be  forwarded  to  the 
sponsoring  organizations  for  their  internal  use. 
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2 .  Survey  #2 

The  data  for  each  individual  item  of  survey  #2  are 
presented  in  summarized  form  in  Appendix  F.  The  qualitative 
responses  relating  to  these  survey  items  are  listed  in 
Appendix  G.  Assessment  items  identified  as  questionable,  that 
is  revealing  a  less  than  99  percent  rating  overall  and  a 
rating  of  less  than  95  percent  in  one  or  more  aspects,  are 
listed  in  Table  3  below.  Deficient  aspects  are  indicated  with 
an  "X". 


Table  3:  Questionable  Assessment  Items. 


3.  Survey  #3 

Survey  #3  data  concerning  the  candidate  self-assessment 
are  summarized  in  Table  4.  The  qualitative  responses  related 
to  items  A4  and  A6  are  listed  in  Appendix  H. 
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Item 

Mean 

Max 

Ala 

1.77 

100 

Alb 

1.81 

100 

A2a 

(0.19) 

0 

A2b 

1.42 

100 

A3 

1.14 

100 

A4 

A5 

45.30 

90 

A6 

Std.  Dev. 


0.48 

88.5% 

0.39 

90.50% 

0.45 

84.00% 

0.66 

71.00% 

0.85 

57.00% 

89.50% 


77.50% 


Qualitative  Response 


50.33% 


Qualitative  Response 


Table  4:  Survey  #3  Data  Summary. 

4 .  Survey  #4 

Survey  #4  data  concerning  the  prototype  software 
application  are  summarized  in  Table  5.  The  responses  to  items 
B6  and  B7  are  listed  in  Appendix  H. 


Mean 


1.81 


0.91 


1.84 


1.47 


1.67 


0.00 


Ranee  Std.  Dev. 


1  0.39 


1  0.29 


3  0.53 


3  0.96 


3  0.68 


0  0.00 


Qualitative  Response 


Qualitative  Response 


90.50% 


91.00% 


92.00% 


73.50% 


83.50% 


100.00% 


Table  5:  Survey  #4  Data  Summary. 


59 


5. 


Interviews 


a.  Project  Manager 

The  project  manager  interview  items  are  in  Appendix 

I.  Individual  responses  to  the  items  are  listed  at  Appendix 

J.  These  responses  are  summarized  as  follows: 

(1)  The  participating  project  offices  intend 
to  use  the  unit  self-assessment  to: 

•  Evaluate  the  overall  health  of  the  organization. 

•  Establish  a  baseline  for  future  comparisons. 

•  Pinpoint  systemic  problem  areas  within  the 
organization. 

•  Establish  organizational  trends. 

(2)  The  types  of  analyses  most  beneficial  to 
the  project  mangers  are: 

•  Trend  analysis  by  major  categories. 

•  Comparison  of  results  by  demographically 
significant  subsets  of  the  project  office. 

•  Benchmarking  against  previous  years  results  and 
against  other  project  offices. 

(3)  The  types  of  presentation  output  most 
beneficial  to  the  project  managers  are: 

•  A  slide  show  presentation,  allowing  the  project 
manager  to  view  the  results  of  the  assessment 
directly  from  the  networked  database  and  print 
selected  analytical  views. 

•  Statistical  charts  summarizing  and  comparing 
analyzed  data. 

•  Bar  charts  summarizing  and  comparing  analyzed  data. 

•  Synopses  of  the  qualitative  response  items. 

(4)  The  project  managers  define  "program 

health"  as: 

•  The  successful  fielding  and  support  of  material  to 
the  soldier. 

•  Mission  accomplishment. 
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•  Personnel  job  satisfaction. 

•  Adequate  personnel  empowerment  and  resourcing. 

(5)  The  project  managers  feel  the  unit  self- 
assessment  is,  for  the  most  part,  focused  on  the  indicators 
that  determine  the  program  health  of  the  organization.  The 
assessment  needs  to  include  focus  on  the  following  areas  to  be 
comprehensive : 

•  Products  and  services. 

•  External  influences  on  the  PMO. 

•  External  and  internal  customer  satisfaction. 

•  Integrated  product  teams. 

(6)  The  project  managers  intend  to  survey  100% 
of  their  respective  organizations  using  the  recommended  self- 
assessment  instrument. 

(7)  The  project  managers  feel  the  demonstrated 
self-assessment  package  will  provide  them  with  the  necessary 
tools  to  identify  weaknesses  in  their  respective  organizations 
and  monitor  their  improvement. 

b.  Unit  Facilitators. 

The  unit  facilitator  interview  items  are  in  Appendix 

K.  Individual  responses  to  the  items  are  listed  at  Appendix 

L.  These  responses  are  summarized  as  follows: 

(1)  The  facilitators  expect  an  automated  unit 
assessment  to  significantly  enhance  their  unit  facilitator 
role.  They  feel  the  assessment  will  reduce: 

•  Facilitator  bias. 

•  The  burden  of  analysis. 

•  The  logistical  burden  associated  with  the 

assessment  process. 

(2)  The  facilitators  do  not  expect  to 
encounter  substantial  difficulties  as  a  result  of  switching  to 
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an  automated  assessment  process.  The  areas  in  which  they 
expect  some  difficulty  are: 

•  Tailoring  of  automated  analyses  to  detect 
demographic  differences. 

•  Determination  of  who,  by  name,  has  not  completed 
the  assessment. 

(3)  The  facilitators  prefer  to  have  the  data 
analysis  and  presentation  fully  automated,  with  minimal 
facilitator  input.  Thus,  enabling  a  consistent,  pre-defined 
statistical  package  to  be  produced  for  each  analysis. 

(4)  The  facilitators  choice  of  analyses  and 
output  match  those  of  their  respective  project  managers. 


C.  ANALYSIS 

This  section  presents  the  analysis  of  the  data  and 
qualitative  responses  summarized  above.  The  analysis  focuses 
on  the  two  major  components  of  the  proposed  unit  self- 
assessment  package:  The  USAREUR  Organizational  Diagnostic 
Survey  and  the  prototype  software  application. 

1.  The  USAREUR  Organizational  Diagnostic  Survey 
(modified) 

The  analysis  of  the  USAREUR  Organizational  Diagnostic 
Survey  is  divided  into  four  areas:  Questionable  assessment 
items;  Instructions  and  procedures;  Length  and  time  required; 
and  applicability  to  a  project  management  office. 

a.  Questionable  Assessment  Items 

Item  1.3.  To  what  extent  are  the  objectives  of  the 
Project  Office  realistic? 

This  item  was  rated  as  irrelevant.  Participants 
rating  this  item  as  irrelevant  can  be  grouped  into  two 
categories.  Several  participants  stated  that  due  to  their 
position  in  the  organization  they  were  unfamiliar  with  the 
project  office's  objectives,  thus  they  had  no  basis  from  which 
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to  judge  their  realism.  These  participants  generally  felt 
that  the  objectives  of  the  project  office  and  this  item  do  not 
apply  to  persons  in  their  positions.  The  second  category  of 
participants  rating  this  item  as  irrelevant  stated  that  the 
objectives  of  the  project  office  are,  in  effect,  determined  by 
management  personnel  and  other  entities  external  to  the 
project  office  itself  and  therefore  should  not  be  addressed  by 
an  internal  assessment  of  the  project  office. 

Although  it  is  imperative  that  all  personnel  within 
a  given  organization  be  familiar  with  that  organization's 
objectives,  as  measured  by  item  1.2,  it  is  clear  that  not  all 
of  these  personnel  are  in  a  position  to  be  able  to  determine 
the  realism  of  these  objectives.  It  is  possible,  however,  for 
all  personnel  within  a  project  office  to  judge  the  realism  of 
the  objectives  of  their  individual  working  group,  branch,  or 
division,  given  the  group' s  present  resources  and  the 
objectives  of  the  next  higher  organization.  To  this  end, 
objectives  for  mission  accomplishment  should  be  formulated  at 
every  level  within  the  organization.  It  is  for  this  reason 
that  the  issue  raised  by  the  second  group  of  participants 
above  is  invalid.  This  item's  focus  is  on  those  objectives 
for  mission  accomplishment  determined  by  the  management  within 
the  project  office.  The  survey  item  is,  however,  too  general 
in  scope  to  allow  all  personnel  within  the  project  office  to 
respond  accurately  without  speculation.  Since  the  purpose  of 
this  survey  is  to  elicit  feedback  from  the  entire 
organization,  this  item,  as  written,  is  irrelevant. 

Item  1.6.  I  am  satisfied  with  my  role  in  setting 
organizational  objectives. 

This  item  was  rated  as  irrelevant.  Participants  rating 
this  item  as  irrelevant  stated  that  persons  of  their  position, 
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as  well  as  many  other  positions  in  the  organization,  do  not 
influence  the  development  of  organizational  objectives. 

Setting  direction  within  an  organization,  to  include 
establishing  organizational  objectives,  is  a  function  of 
management.  Therefore,  a  majority  of  the  individuals  within 
a  project  office  would  not  be  expected  to  play  a  role  in 
accomplishing  this  task.  Since  this  survey  is  designed  to 
apply  to  all  personnel  within  the  project  office,  this  item  is 
irrelevant . 

Item  1.7.  To  what  extent  are  the  Project  Office's 
objectives  for  mission  accomplishment  consistent  with  the 
demands  of  its  environment? 

This  item  was  rated  as  unclear,  wordy,  ambiguous, 
and  irrelevant.  Participants  rating  this  item  as  unclear, 
wordy,  and  ambiguous  stated  they  did  not  understand  the  phrase 
"the  demands  of  its  environment"  and  therefore  also  did  not 
understand  what  type  of  response  was  being  solicited. 
Participants  indicating  the  item  was  irrelevant  stated  the 
"objectives  for  mission  accomplishment"  did  not  pertain  to 
person  in  their  position  within  the  organization. 

The  phrase  "the  demands  of  its  environment"  implies 
a  potentially  infinite  number  of  variables.  This  phrase  also 
presupposes  that  all  personnel  within  the  organization  are 
familiar  with  the  organization's  environment  and  the  demands 
placed  on  the  organization  by  external  entities.  Thus,  this 
item,  as  written,  lacks  sufficient  definition  to  solicit  a 
singular  response  and  therefore  is  both  unclear  and  ambiguous. 
As  with  item  1.3,  this  item  requires  the  majority  of  personnel 
within  the  project  office  to  speculate  as  to  the 
appropriateness  of  the  project  office's  objectives.  As 
discussed  above,  many  persons  within  the  organization  are  not 
in  a  position  to  respond  to  items  of  this  type  with  any 
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significant  degree  of  accuracy,  thus  this  question  is  also 
irrelevant . 

Item  1.8.  To  what  extent  are  the  Project  Office's 
objectives  for  mission  accomplishment  consistent  with  the 
availability  of  its  resources? 

This  item  was  rated  as  unclear,  wordy,  and 
ambiguous.  Participants  rating  the  item  in  this  manner  stated 
they  did  not  understand  the  phrase  "the  availability  of  its 
resources"  and  therefore  also  did  not  understand  what  an 
appropriate  response  should  be. 

This  item,  as  in  the  case  of  item  1.7,  presupposes 
knowledge  of  the  project  office's  available  resources  and 
requires  a  stricter  definition  of  the  term  resources  to  enable 
personnel  within  the  organization  to  respond  accurately. 
Thus,  this  item,  as  written,  is  unclear,  wordy,  and  ambiguous. 

Item  1.10.  The  strategy  of  the  Project  Office  could 
be  improved  by: 

This  item  was  rated  as  unclear  and  ambiguous. 
Participants  who  indicated  that  this  item  was  unclear  or 
ambiguous  were  troubled  by  the  word  "Strategy".  They  either 
did  not  fully  understand  what  an  organizational  strategy  was 
or  they  were  not  familiar  with  the  strategy  of  their 
respective  organization. 

This  rating  is  understandable,  given  that  this 
survey  is  intended  for  all  members  of  the  organization. 
Strategy  development  is  clearly  a  function  of  management. 
Personnel  other  than  management  are  therefore  not  expected  to 
be  in  a  position  to  comment  on  strategy  improvement.  All 
personnel,  however,  should  be  knowledgeable  as  to  what  the 
organization's  strategies  are.  This  item,  although  clear  and 
unambiguous,  is  inappropriate  for  this  survey. 
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Item  2.2.  To  what  extent  do  your  assigned  tasks 
include  work  that  you  consider  to  be  outside  your  area  of 
responsibility? 

This  item  and  its  answer  scale  were  rated  as 
inappropriate.  Participants  rating  this  item's  answer  scale 
as  inappropriate  stated  the  scale  did  not  provide  appropriate 
responses  to  answer  the  item.  They  suggested  the  item  be 
reworded  as  a  statement,  using  the  Agree/Disagree  scale.  More 
importantly,  this  item  was  viewed  by  several  leaders  within 
the  organization  as  a  source  of  potential  grievances  due  to 
its  wording.  For  this  reason  alone  the  item  is  inappropriate. 

Item  2.3.  To  what  extent  are  your  assigned  task 
deadlines  reasonable? 

This  item  was  rated  as  unclear.  Participants  rating  this 
item  as  unclear  felt  the  item  lacked  sufficient  definition. 
One  group  of  participants  stated  the  item  should  be  focused  on 
normal  or  routine  tasks,  eliminating  those  tasks  associated 
with  the  natural  level  of  inherent  crisis  management 
associated  with  the  project  management  discipline.  Another 
group  of  respondents  stated  the  word  "reasonable"  was  not 
adequately  defined. 

Although  there  is  a  certain  amount  of  crisis 
management  involved  in  the  normal  operation  of  all 
organizations,  there  is  not  sufficient  rationale  to  discount 
this  fact  in  this  items  wording.  In  fact,  it  is  this  level  of 
crisis  management  that  the  item  is  attempting  to  measure.  To 
what  extent  does  the  management  of  the  project  office  rely  on 
crisis  management  techniques,  rather  than  proactive  planning? 
Second,  "reasonable"  is  inherently  defined  as  the  participants 
opinion  as  to  what  is  reasonable  and  therefore  it  would  be 
constraining  the  survey  participants  by  further  defining  this 
term.  This  item  is  clear  as  written. 
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Item  3.1.  To  what  extent  does  the  Project  Office's 
present  structure  facilitate  work? 

This  item  was  rated  as  unclear.  The  participants 
indicating  this  item  was  unclear  stated  that  the  word  "work" 
was  not  adequately  defined. 

This  researcher  agrees  that  the  word  "work"  in  this 
context  appears  unbounded.  This  item  should  be  rewritten  to 
focus  on  the  extent  the  project  office's  present  structure 
facilitates  or  hinders  the  participants  ability  to  accomplish 
his/her  assigned  tasks.  This  item,  as  written,  is  unclear. 

Item  3.3.  To  what  extent  does  the  Project  Office's 
present  structure  facilitate  decision  making? 

This  item  was  rated  as  ambiguous.  Several 
participants  indicated  this  item  was  ambiguous  stating  that 
the  level  of  decision  making  being  considered  needs  to  be 
defined. 

This  researcher  agrees  that  the  level  of  decisions 
being  considered  is  an  important  element  to  be  included  in 
this  item.  This  item  should  focus  at  a  level  at  which  all 
personnel  within  the  organization  can  respond  to  the  item 
without  speculation.  This  item,  as  written,  is  ambiguous. 

Item  3.4.  Our  informal  work  structure  complements 
the  established  formal  structure. 

This  item  was  rated  as  unclear.  Participants 
indicating  this  item  was  confusing  did  not  understand  the 
references  to  the  "informal  work  structure"  and  "formal 
structure" . 

It  is  apparent  to  this  researcher  how  these  terms 
could  be  a  source  of  confusion,  given  the  structure  and 
environment  of  the  military  project  management  office.  Within 
the  overall  structure  of  the  project  office  there  are  multiple 
formal  and  informal  sub-structures.  First,  there  is  the 
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traditional  line  authority  structure  of  the  project  office. 
This  structure  does  not,  however,  include  all  personnel 
dedicated  to  the  project.  Many  of  the  personnel  directly 
supporting  the  project  office  are  matrix  personnel.  That  is, 
they  are  assets  belonging  to  other  organizations  that  are 
dedicated  to  and  work  within  the  project  office.  Each  of 
these  individuals  also  reports  to  superiors  in  their  parent 
organization.  Second,  much  of  the  work  in  the  project  office 
is  accomplished  within  Integrated  Product  Teams  (IPT) .  These 
IPTs  are  informal  structures  developed  to  facilitate  the 
efficient  interaction  between  all  functional  areas  involved  in 
the  production  of  a  particular  product.  Finally,  there  is 
usually  an  ad  hoc  structure  developed  by  the  workers  in  an 
organization  that  is  used,  as  necessary,  in  lieu  of  the  stated 
formal  structure  to  accomplish  required  tasks  in  the  most 
efficient  manner.  Depending  on  the  position  and  parent 
organization  of  the  survey  participant  it  could  be  unclear  to 
which  formal  or  informal  structure  this  item  is  referring. 
This  item,  as  written,  is  unclear. 

Item  3.5.  To  what  extent  are  individual 
responsibilities  defined  in  the  Project  Office? 

This  item  was  rated  as  unclear  and  ambiguous. 
Personnel  indicating  this  item  was  confusing  felt  this  item 
was  too  general  and  asked  for  information  that  cannot  be 
obtained  from  personal  knowledge. 

This  researcher  agrees  that  this  item  solicits  a 
speculated  response.  This  item  should  be  worded  to  refer  only 
to  the  participant's  responsibilities.  This  item,  as  written 
is  unclear  and  ambiguous. 

Item  6.3.  To  what  extent  is  conflict  dealt  with 
openly  and  managed  constructively  in  the  Project  Office? 
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This  item  was  rated  as  unclear.  Participants  rating 
this  item  as  unclear  stated  that  the  item  was  confusing 
because  the  types  and  level  of  conflict  are  undefined. 

This  researcher  agrees  that  the  types  and  levels  of 
conflict  being  measured  are  critical  components  of  the  item. 
Within  any  organization  there  is  a  broad  range  of  conflict 
types,  such  as  personal  conflicts,  conflicting  tasks, 
conflicts  between  superiors  and  subordinates,  etc.  The  type 
of  conflict  and  the  level  at  which  it  occurs  in  the 
organization  must  be  clearly  defined  to  enable  the  survey 
participant  to  accurately  respond  to  this  item.  This  item,  as 
written,  is  unclear. 

b.  Instructions  and  Procedures 

The  instructions  provided  on  how  to  complete  the 
assessment  were  relatively  complete  and  clear  to  most  of  the 
survey  participants.  Of  the  participants  indicating  the 
instructions  were  incomplete,  most  referred  to  a  lack  of 
guidance  in  the  use  of  the  provided  software  and  not  the 
survey  instrument  itself.  This  is  a  separate  issue  discussed 
in  the  analysis  of  the  prototype  software.  Those  participants 
stating  the  instructions  were  confusing  indicated  that  the 
reference  to  the  types  of  items  included  in  the  survey  were 
confusing.  The  survey  instructions  refer  to  items  soliciting 
a  qualitative  response  as  "free"  or  "open"  response  items  and 
items  requiring  the  participant  to  select  an  appropriate 
response  from  a  fixed  set  of  responses  as  "fixed"  response 
items.  This  shortcoming  is  easily  reconciled  by  using  more 
traditional  terms,  such  as  "short  answer"  and  "multiple 
choice"  to  refer  to  the  item  types.  Once  the  instructions 
were  clarified,  none  of  the  participants  encountered  any 
significant  difficulties  associated  with  the  survey 
procedures.  Again,  those  difficulties  that  did  surface  were 
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connected  to  the  use  of  the  software  and  were  not  linked  to 
the  assessment  instrument. 

c.  Length  and  Time  Required 

The  candidate  survey  consisted  of  74  items,  compared 
to  66  items  incorporated  by  the  Self-assessment  Guide  for 
Organizational  Performance  and  Customer  Satisfaction.  The 
majority  of  the  survey  participants  indicated  that  the  length 
of  the  survey  was  adequate  to  ensure  a  comprehensive 
examination  of  the  organization' s  program  health  and  customer 
satisfaction  without  requiring  an  excessive  amount  of  time  to 
complete.  Several  participants  did  indicate  that  they  felt 
the  survey  contained  too  many  items,  however  the  time  required 
to  complete  the  survey  was  acceptable. 

The  average  time  required  for  each  participant  to 
complete  the  survey  was  44  minutes.  The  44  minute  average  time 
to  complete  the  survey  is  inflated  due  to  each  participant 
providing  feedback  on  the  survey  items  as  they  completed  the 
survey.  This  researcher  estimates  that  an  average  of  ten 
minutes  of  the  time  required  for  each  participant  to  complete 
the  survey  was  dedicated  to  providing  this  feedback.  It  is 
therefore  estimated  that  the  required  time  to  complete  this 
survey,  on  average,  is  34  minutes.  This  is  compared  to  an 
estimated  time  of  90  minutes  necessary  to  complete  the  Self- 
assessment  Guide  for  Organizational  Performance  and  Customer 
Satisfaction. 

Although  the  candidate  survey  contains  eight  items 
more  than  the  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction,  it  is  significantly 
shorter  and  requires  less  time  to  complete  than  its 
predecessor.  There  are  two  reasons  supporting  this 
conclusion.  First,  although  the  Self-assessment  Guide  for 
Organizational  Performance  and  Customer  Satisfaction  contains 
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only  66  items,  the  responses  provided  for  these  items  are 
lengthy  sentences  or,  in  some  cases,  paragraphs.  This 
requires  the  participant  to  dedicate  a  substantial  amount  of 
his/her  time  to  read  each  response  prior  to  answering  the 
items.  Highlighting  this  is  the  fact  that  the  66  items  of  the 
Self-assessment  Guide  for  Organizational  Performance  and 
Customer  Satisfaction  span  44  pages  of  text,  whereas  the  74 
items  of  the  candidate  assessment  are  covered  in  six  pages. 
Second,  the  candidate  assessment  incorporates  an  automated 
approach,  allowing  the  participant  to  quickly  view  the  item, 
indicate  his/her  response  and  continue  to  the  next  item, 
whereas  the  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction  does  not. 

d.  Applicability  to  a  Project  Management  Office 
The  majority  of  the  survey  participants  indicated 
that  the  candidate  assessment,  as  a  whole,  is  applicable  to  a 
project  management  office.  These  participants  felt,  however, 
that  many  of  the  survey  items  were  not  appropriate  for  persons 
of  their  position  within  the  organization.  The  main  reason 
cited  for  this  observation  was  that  the  scope  of  these  items 
were  too  broad  and  included  elements  outside  of  their  purview. 
This  researcher  whole  heartedly  agrees  with  this  observation. 
In  fact,  this  observation  applies  to  the  majority  of  the  items 
included  in  the  assessment.  This  observation  is  addressed  in 
detail  in  the  discussion  section  of  this  chapter. 

2.  Prototype  Software  Application 

The  analysis  of  the  prototype  software  application  is 
divided  into  two  areas:  Instructions  and  procedures;  and  Ease 
of  use  and  time  required. 

a.  Instructions  and  Procedures 

The  majority  of  the  survey  participants  indicated 
that  the  software  instructions  were  comprehensive  and  clearly 


71 


stated.  The  only  difficulty  associated  with  the  instructions 
involved  identifying  particular  items  as  they  actually  appear 
within  the  software,  based  on  their  description  in  the 
instructions.  This  shortcoming  is  addressed  in  the  first 
production  version  of  the  software  by  including  the 
instruction  within  the  application' s  on-line  help  and 
providing  graphical  examples  of  the  items,  screens,  and  other 
objects  unique  to  the  software  application. 

b.  Ease  of  Use  and  Time  Required 

All  of  the  survey  participants,  without  exception, 
stated  that  the  software  application  was  "user  friendly"  and 
that  they  encountered  only  minor  difficulties  completing  the 
survey  using  the  application.  All  of  the  participants,  except 
one,  indicated  that  the  Microsoft  Windows  environment  was 
installed  on  their  workstation  and  therefore  the  majority  of 
the  procedures  and  items  incorporated  into  the  software 
application  were  already  familiar  to  them.  The  participants 
stated  that  the  automating  of  the  assessment  significantly 
enhanced  the  overall  assessment  process,  requiring  them  less 
time  to  complete  the  assessment  than  a  manual  system  and 
providing  them  with  increased  flexibility  and  comfort. 

D.  DISCUSSION 

The  discussion  of  the  issues  related  to  the  proposed  unit 
self-assessment  package,  as  in  the  analysis  section,  is 
focused  on  the  packages  two  major  components:  The  USAREUR 
Organizational  Diagnostic  Survey  and  the  prototype  software 
application. 

1.  The  USAREUR  Organizational  Diagnostic  Survey 
(modified) 

a.  Assessment  Items 

In  addition  to  the  items  identified  as  questionable 
in  the  analysis  section  of  this  chapter,  numerous  other  items 
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in  the  candidate  assessment  contain  shortcomings  identical  to 
those  identified  with  these  items.  These  items  fall  into 
three  categories: 

•  Items  to  which  all  participants  cannot  accurately 
respond  without  significant  speculation  because  the 
item  is  focused  at  too  high  a  level  within  the 
organization  (i.e.  project  office  rather  than 
section)  or  the  scope  is  unbounded. 

•  Items  which  requires  participants  to  infer  the 
opinions  of  other  people  within  the  organization. 

•  Items  in  which  the  subject  matter  limits  meaningful 
responses  to  a  select  group  of  people  within  the 
organization. 

The  first  category,  items  focused  at  too  high  a 
level  within  the  organization,  applies  to  a  significant  number 
of  the  items  included  in  the  candidate  survey.  Items  within 
this  category  solicit  generalized  responses  from  participants 
that  require  them  to  draw  conclusions  about  the  entire 
organization.  These  items  should  be  focused  at  the  lowest 
organized  level  within  the  organization  to  allow  participants 
to  provide  accurate  responses  concerning  these  items  from  a 
individual  perspective,  without  speculation.  Trends  and 
conclusions  concerning  the  entire  organization  are  then 
revealed  by  combining  these  individual  responses  into  an 
aggregate  information  database.  Items  included  in  this 
category  are: 


Item 

1.2 

Item 

4.2 

Item 

6.8 

Item 

7.7 

Item 

1.9 

Item 

4.5 

Item 

6.9 

Item 

7.8 

Item 

2.8 

Item 

4.6 

Item 

7.1 

Item 

8.1 

Item 

2.9 

Item 

4.7 

Item 

7.2 

Item 

8.3 

Item 

3.1 

Item 

6.2 

Item 

7.3 

Item 

8.5 

Item 

3.10 

Item 

6.3 

Item 

7.4 

Item 

9.1 

Item 

3.11 

Item 

6.5 

Item 

7.5 

Item 

9.2 

Item 

4.1 

Item 

6.6 

Item 

7.6 

Item 

9.3 

The  CCAWS/AGMS  Unit  Self-assessment  reflects  adjustments  made 
to  these  items  with  regard  to  the  organizational  level  at 
which  these  items  are  focused. 
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The  second  category,  items  that  require  participants 

to  infer  the  opinions  of  other  people,  includes: 

Item  1.5  Item  3.8  Item  5.4  Item  6.4 

Item  3.5  Item  4.3  Item  5.6  Item  6.7 

Item  3.6  Item  4.4  Item  6.1  Item  8.4 

Item  3.7 

These  items  have  also  been  adjusted  in  the  CCAWS/AGMS  Unit 
Self-assessment  to  permit  participants  to  respond  to  each  item 
based  on  individual  knowledge. 

The  final  category  above,  items  whose  subject  matter 


limit  accurate 

responses  to 

a  select  group 

of  people, 

includes : 

Item  1.3 

Item  1.7 

Item  3.3 

Item  8.2 

Item  1.4 
Item  1 . 6 

Item  1.8 
Item  1.10 

Item  5.1 

Item  8.7 

These  items  do  not  meet  the  intent  of  the  survey  design  and 
have  been  omitted  from  the  CCAWS/AGMS  Unit  Self-assessment. 


b.  Functional  Areas  and  Model 

The  USAREUR  Organizational  Diagnostic  Survey 
(modified)  evaluated  during  the  prototype  testing  consisted  of 
eight  functional  areas,  described  in  Chapter  IV,  and  a  general 
category.  Based  on  the  qualitative  responses  gathered  during 
the  evaluation  of  the  candidate  assessment  and  from  interviews 
conducted  with  the  project  and  deputy  project  managers  of  the 
sponsoring  organizations,  it  is  evident  that  the  candidate 
assessment,  although  a  more  appropriate  solution  than  the 
Self-assessment  Guide  for  Organizational  Performance  and 
Customer  Satisfaction,  does  not  effectively  address  all  of  the 
critical  factors  that  determine  the  program  health  and 
customer  satisfaction  of  a  project  management  office. 
Specifically,  the  candidate  assessment  addresses  two 
functional  areas  determined  by  the  project  managers  to  be 
inconsequential  to  the  program  health  and  customer 
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satisfaction  of  a  project  office  and  fails  to  address  three 
additional  areas,  cited  by  these  managers  as  critical  factors. 

The  two  areas  recommended  for  deletion  from  the 
assessment  were  technology  and  rewards.  Technology  was  said 
to  be  inconsequential  because  the  sponsoring  project  offices 
are  fully  automated  and  there  is  no  apparent  weaknesses  in 
this  area.  This  researcher  disagrees  with  this  rationale  for 
the  following  reasons.  First,  although  each  of  the  sponsoring 
project  offices  may  be  considered  fully  automated  and  void  of 
shortcomings  in  this  area  at  present,  this  assessment  is 
designed  to  be  conducted  annually  for  an  undetermined  period. 
Given  the  speed  at  which  technological  advancements  are 
forthcoming,  the  technological  state  at  which  these  two 
organizations  are  assessed  could  significantly  deteriorate 
from  one  period  to  the  next.  Second,  the  context  in  which 
technology  is  used  by  these  managers  and  in  the  candidate 
assessment  refers  only  to  the  equipment  available  to  the 
project  offices.  In  reality,  technology  encompasses  not  only 
this  physical  equipment,  but  also  the  basic  methods  and 
techniques  used  to  maximize  equipment  usage.  Therefore,  the 
scope  of  this  functional  area  should  be  broadened  and  the 
items  within  this  area  reviewed  to  reflect  this  definition  of 
technology. 

The  rewards  item  group  was  recommended  for  deletion 
because  rigid  policies  directed  by  authorities  external  to  the 
project  office  limit  the  latitude  of  managers  within  the 
project  office  to  effect  change  in  this  area.  Although  this 
is  true,  in  part,  leaders  within  these  organizations  retain 
the  authority  to  ensure  rewards  are  equitably  distributed 
within  these  guidelines.  These  guidelines  do,  however, 
minimize  this  areas  effect  to  attribute  to  the  overall  program 
health  of  the  project  office.  Therefore,  the  critical  items 
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remaining  in  this  area  can  be  merged  into  the  people  category 
and  this  area  deleted  from  the  assessment. 

The  factors  critical  to  the  program  health  and 
customer  satisfaction  of  a  project  management  office  that  were 
not  addressed  by  the  candidate  assessment  are:  Outputs  (i.e. 
products  and  services),  the  Integrated  Product  Team  (IPT) ,  and 
Customer  Satisfaction. 

Given  the  addition  and  deletion  of  the  areas 
described  above,  a  new  model  was  developed  depicting  the 
factors  critical  to  the  assessment  of  program  health  and 
customer  satisfaction  of  a  project  management  office  (Figure 
IV)  . 
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Figure  17:  The  CCAWS/AGMS  Unit  Self-assessment  Model. 
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This  model  accurately  depicts  the  inputs  associated  with  the 
military  project  management  environment  and  reflects  the 
increased  importance  of  the  organizational  vision  and  the 
incorporation  of  multi-organizational  design  factors  used  in 
the  Integrated  Product  Team. 

Comparing  this  model  to  that  of  the  Harvard  Business 
School,  it  is  clear  that  in  addition  to  accurately  reflecting 
the  critical  factors  that  determine  program  health  and 
customer  satisfaction  in  a  project  office,  the  CCAWS/AGMS  Unit 
Self-assessment  model  sufficiently  addresses  the 
organizational  design  factors  identified  as  critical  by  the 
Harvard  School  of  Business  (Figure  18) . 
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Figure  18:  A  Comparison  of  the  CCAWS/AGMS  Unit  Self- 
assessment  Model  and  the  Harvard  Business  School  Model. 
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c.  Program  Health  and  Customer  Satisfaction 

The  candidate  assessment  instrument,  in  its  final 
form,  is  appropriate  and  effective  to  measure  the  program 
health  and  internal  customer  satisfaction  of  a  project 
management  office.  This  instrument,  however,  does  not 
sufficiently  measure  the  satisfaction  of  customers  external  to 
the  project  office  itself. 

2.  Prototype  Software  Application 
a .  Computer  Resources 

The  prototype  software  application  is  compatible 
with  the  current  computer  resources  used  in  the  CCAWS  and  AGMS 
project  offices.  Both  project  offices  are  running 

International  Business  Machines  (IBM)  or  compatible 
workstations.  Each  of  these  workstations  has  the  equivalent 
of  a  38 6DX-25  or  faster  Central  Processing  Unit  (CPU)  with  a 
minimum  of  four  megabytes  of  Random  Access  Memory  (RAM)  .  The 
workstations  are  operated  by  a  Microsoft  Disk  Operating  System 
(MS-DOS),  version  5.0  or  greater,  in  the  MS  Windows  3.1  or 
3.11  environment.  Monitors  at  these  workstations  contain 
video  cards  capable  of  supporting  resolutions  of  640x480  or 
800x600  pixels.  The  minimum  workstation  requirements  for  use 
with  the  user  interface  application  are:  CPU,  386SX-20;  RAM, 
MB;  MS-DOS,  version  5.0;  MS  Windows,  version  3.1;  video 
resolution,  640x480.  Each  of  the  sponsoring  organizations  use 
a  Local  Area  Network  (LAN)  manufactured  by  the  Novell 
corporation.  The  user  interface  is  designed  to  operate  on  a 
Novell  LAN,  however,  compatibility  tests  with  the  LANs  in  the 
CCAWS  and  AGMS  project  offices  was  not  conducted  as  part  of 
the  prototype  software  evaluation. 

The  data  storage,  analysis,  and  presentation 
application  is  designed  to  operate  within  Microsoft  Access, 
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version  6.0.  This  program  is  currently  being  used  by  both  the 
CCAWS  and  AGMS  project  offices. 

b.  User  Interface 

The  user  interface  application  performed  remarkably 
well  during  the  prototype  testing.  The  application  compiled 
data  from  50  respondents  with  no  significant  failures.  There 
were,  however,  several  minor  shortcomings  associated  with  the 
application  that  are  characterized  as  normal  occurrences 
during  the  beta  testing  of  a  new  software  application.  These 
shortcomings  and  their  solutions  are  discussed  below. 

(1)  Method  of  Indicating  Responses.  At  the 
time  of  testing,  the  user  interface  provided  each  respondent 
with  two  methods  to  indicate  their  response  to  fixed 
assessment  items.  The  respondents  could  either  drag  the 
indicator  up  and  down  the  answer  scale  using  his/her  pointing 
device  (mouse)  or  perform  the  same  maneuver  using  the  arrow 
keys  on  the  keyboard.  During  testing  this  researcher 
observed,  however,  that  the  first  inclination  of  each 
respondent,  with  very  few  exceptions,  was  to  attempt  to 
indicate  his/her  response  by  pointing  at  the  desired  response 
itself  with  the  pointing  device.  This  was  predominately  the 
case,  even  though  each  respondent  had  read  the  software 
instructions  detailing  the  two  included  procedures.  As 
respondents  attempted  to  indicate  their  responses  in  this 
fashion,  to  which  the  application  did  not  respond,  each  became 
confused  and  either  referred  to  the  software  instructions  or 
prompted  the  researcher  for  assistance.  Since  one  of  the 
primary  goals  in  developing  the  user  interface  application  was 
to  simplify  the  assessment  process  by  providing  a  "user 
friendly"  interface  for  data  collection,  this  researcher 
modified  the  interface  application  to  enable  respondents  to 
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indicate  their  responses  by  pointing  at  the  desired  response, 
as  was  their  propensity  to  do  so. 

(2)  File  Structure.  The  user  interface 
application  at  the  time  of  testing  consisted  of  ten  separate 
executable  software  programs,  a  database  control  and  nine 
functional  areas,  and  their  associated  Dynamic  Link  Library 
(DLL)  files.  Although  this  arrangement  simplified  the 
packaging  and  transportation  of  the  application,  it  created 
considerable  problems  in  the  functioning  of  the  application. 
First,  because  each  functional  area  was  contained  in  a 
separate  executable  file,  each  time  a  respondent  entered  into 
a  new  functional  area  (e.g.  Strategy  to  Task)  a  significant 
pause  was  encountered,  approximately  five  seconds,  as  one 
executable  program  was  opened  and  the  other  closed.  These 
delays  created  considerable  confusion  with  each  respondent. 

Second,  since  the  database  control  function  was 
located  in  a  separate  executable  file  than  each  functional 
area,  difficulties  were  encountered  identifying  the  correct 
database  record  within  each  functional  area  that  data  were  to 
be  written  to  or  stored  in.  At  the  time  of  testing  this  was 
accomplished  through  the  use  of  an  external  file,  created  by 
the  database  function,  containing  the  respondent 
identification  number.  Each  functional  area,  once  opened, 
queried  the  contents  of  this  file  to  determine  the  record  in 
which  the  data  were  to  be  stored.  Although  this  technique 
performed  successfully  during  the  prototype  testing,  the 
testing  was  conducted  on  a  stand  alone  system.  Once  the 
application  is  loaded  onto  the  organization's  Local  Area 
Network  (LAN)  this  will  not  be  the  case.  When  the  application 
is  used  on  a  LAN  there  is  a  high  probability  that  two  or  more 
respondents  will  access  the  application  at  the  same  time.  If 
this  situation  occurs,  the  first  respondent  to  complete 
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entering  his/her  demographic  data  and  begin  the  assessment 
will  have  his/her  respondent  number  recorded  to  the  external 
file.  Again,  this  file  is  used  by  successive  programs  to 
determine  his/her  record  in  the  database.  Once  the  second 
person  begins  the  assessment,  however,  the  application  will 
record  his/her  respondent  identification  number  to  the 
external  file,  overwriting  the  first  person's  data.  After 
that  time  when  the  first  person  moves  from  one  functional  area 
to  another,  the  external  file  will  provide  the  current  or 
updated  information  back  to  the  application,  thus  data  entered 
by  the  first  person  would  be  inadvertently  stored  in  the 
second  person's  database  record,  destroying  both  sets  of  data. 

Each  of  the  above  surfaced  anomalies  were 
corrected  by  restructuring  the  application's  program  files. 
The  application' s  first  production  version  consists  of  one 
executable  file,  containing  both  the  database  control  and  all 
functional  areas,  and  its  associated  DLL  files.  This 
structure  allows  a  seamless  transition  from  one  functional 
area  to  the  next  and  enables  the  database  function  to 
accurately  track  the  respondents  record  identification  number 
throughout  the  assessment  process. 

(3)  Exporting  Data.  One  of  the  sponsor's 
automation  requirements  was  that  the  recommended  software 
application  be  compatible  with  the  existing  computer  resources 
available  within  their  organizations.  Since  the  database 
function  included  in  Novell's  AppWare  development  program 
stores  data  to  a  unique  database  file,  it  was  essential  to 
export  data  from  this  file  to  a  common  or  standard  format.  Of 
the  standard  data  formats  available,  the  two  most  common  are 
the  comma  delimited  and  the  tab  delimited.  The  comma 
delimited  format  uses  commas  to  separate  data  elements  within 
each  data  record.  The  tab  delimited  performs  the  same 
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operation  using  tabs.  Data  stored  in  either  of  these  two 
formats  can  be  readily  imported  into  Microsoft  Access  and 
Excel,  used  by  the  sponsoring  organizations,  or  a  number  of 
other  popular  database,  spreadsheet,  and  analysis  programs. 
Of  these  two  formats  this  researcher  chose  the  comma  delimited 
format  for  use  in  the  prototype  application.  During  testing, 
however,  it  was  noticed  that  many  of  the  respondents  preferred 
the  use  of  commas  to  clarify  statements  when  answering  the 
open  response  items.  When  exporting  these  data  into  a  comma 
delimited  data  format,  the  application  interprets  the  commas 
internal  to  each  open  response  item  as  the  beginning  of  a  new 
response  (i.e.  a  response  including  three  commas  is 
interpreted  as  four  responses)  .  Therefore,  when  the  data  are 
imported  into  the  storage,  analysis,  and  presentation 
application,  all  responses  following  an  open  response  with 
internal  commas  are  misaligned  by  the  number  of  commas 
included  in  the  defective  response.  This  occurrence  would 
corrupt  the  data  included  in  that  particular  record  and 
invalidate  any  analyses  performed  on  the  entire  set  of  data. 

This  anomaly  has  also  been  corrected  in  the 
first  production  version  of  the  software  application.  The 
production  version  incorporates  the  tab  delimited  data  format 
to  export  data  for  use  in  the  data  storage,  analysis,  and 
presentation  application.  This  format  will  enable  respondents 
to  use  any  and  all  characters  in  their  open  item  responses 
without  interfering  with  the  data  export  function. 

(4)  Next/Previous  Item  Function.  At  the  time 
of  testing,  the  user  interface  application  incorporated  push 
buttons  to  enable  participants  to  move  from  one  assessment 
item  to  the  following  item.  During  testing,  a  majority  of  the 
participants  revealed  a  strong  desire  to  return  to  items 
answered  previously  to  review  or  compare  their  answers.  In 
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response  to  this  feedback,  push  buttons  have  been  inserted 
into  each  item  screen  to  allow  participants  to  move  both 
forwards  and  back  through  the  available  items. 

(5)  Resume  Function.  The  user  interface 
application  incorporates  a  resume  function  that  allows  the 
participant  to  discontinue  the  assessment  and  finish  the  same 
assessment  at  a  later  time.  At  the  time  of  testing,  this 
function  required  that  the  participant  had  written  down  or 
remembered  his/her  respondent  identification  number,  assigned 
at  the  beginning  of  the  assessment.  This  function  was  also 
limited  in  the  fact  it  would  only  allow  participants  to  return 
to  the  first  item  in  the  desired  functional  area.  During 
testing,  it  was  revealed  that,  without  prompting  by  the 
researcher,  participants  did  not  understand  the  significance 
of  remembering  their  respondent  identification  number  until 
after  they  attempted  to  resume  the  assessment.  The  production 
version  of  the  software  application  addresses  this  shortcoming 
by  including  a  screen  that  appears  to  the  participant  when 
he/ she  attempts  to  terminate  the  assessment  process  at  any 
point  other  than  the  final  item.  This  screen  will  provide  the 
participant  with  his/her  respondent  identification  number  and 
the  number  of  the  last  item  completed  and  prompt  the 
participant  to  record  this  information  for  later  use. 

(6)  Demographic  Lists.  The  user  interface 
application  queries  each  participant  for  demographic  and  other 
essential  information  using  a  set  of  fixed  "drop  down"  lists. 
This  information  is  used  by  the  analysis  and  presentation 
application  to  group  data  for  analytical  purposes.  Listings 
used  in  the  application  were  compiled  using  information 
provided  by  the  CCAWS  and  AGMS  project  offices.  The 
categories  represented  are:  Year;  Organization;  Office; 
position;  time  in  organization,  office,  position;  Age;  Sex; 
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Military/Civilian;  Rank/Grade;  and  Education  level.  Despite 
the  attempt  to  compile  a  comprehensive  listing  of  the 
applicable  items  within  each  of  these  categories,  it  was 
revealed  during  the  prototype  testing  that  some  of  the  lists 
were  incomplete.  An  attempt  at  a  one  time  fix  to  this  problem 
would  most  probably  also  end  in  failure.  Therefore,  the 
production  version  of  the  user  interface  application 
incorporates  a  function  that  will  enable  the  unit  facilitator 
to  add  items  to  each  of  these  lists  during  "run  time",  as 
necessary. 

c.  Data  Storage,  Analysis,  and  Presentation 

The  data  storage,  analysis,  and  presentation 
application  was  not  available  during  the  prototype  software 
testing  in  June  1995.  This  application  was,  however, 
demonstrated  for  the  CCAWS  Project  Manager  in  August  1995. 
Results  of  this  demonstration  and  comments  from  the  unit 
facilitators  and  the  deputy  project  managers  from  both 
organizations  as  to  the  design  of-  this  application  are 
discussed  below. 

(1)  Data  Storage.  The  sponsors'  requirements 
to  automate  the  assessment  process  included  a  data  storage 
function  capable  of  storing  collected  data  for  a  period  of  not 
less  than  five  years.  Both  the  user  interface  and  the  data 
storage,  analysis,  and  presentation  applications  meet  this 
requirement.  The  user  interface,  although  designed  for 
temporary  storage  of  data  during  the  assessment  process,  is 
limited  only  by  the  number  of  years  recorded  in  the  year  list 
within  the  application.  Since  this  list  can  be  updated  by  the 
unit  facilitator,  as  necessary,  the  capacity  of  the  user 
interface  to  store  data  for  the  purposes  of  the  unit  self- 
assessment  is  unlimited.  The  application,  as  delivered, 
provides  an  initial  capability  to  store  12  years  of  data.  The 
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data  storage,  analysis,  and  presentation  application  is 
designed  for  the  permanent  storage  of  collected  data.  Its 
capacity  to  accomplish  this  task,  as  delivered,  is  unlimited. 
It  is  important  to  note,  however,  that  data  modified  directly 
from  within  the  data  storage,  analysis,  and  presentation 
application  after  they  are  initially  stored  must  be  saved  to 
a  separate  file.  Although  these  data  can  still  be  used,  once 
data  from  the  next  assessment  iteration  are  imported  from  the 
user  interface,  these  data  will  be  overwritten  and  all  changes 
to  the  data  made  in  the  data  storage,  analysis,  and 
presentation  application  will  be  lost.  This  function  was 
purposely  incorporated  to  purge  the  database  of  duplicate 
records.  The  function  does  not  compare  data  records  to  locate 
duplicates,  rather  it  keys  on  the  dates  the  data  were  entered. 
Each  data  record  is  date  stamped  as  it  is  imported  into  the 
permanent  database.  As  new  data  are  imported,  the  application 
purges  all  records  with  a  date  stamp  that  is  earlier  than  the 
import  date.  This  process  does  not  permanently  erase  previous 
years  data,  as  they  were  originally  entered,  because  these 
data  are  again  imported  into  the  data  storage,  analysis,  and 
presentation  application  along  with  the  new  data  records. 

(2)  Analysis.  The  data  storage,  analysis,  and 
presentation  application  is  designed  to  automatically  format, 
store,  query,  and  perform  calculations  on  data  as  they  are 
imported  into  the  permanent  database.  Calculations  performed 
on  the  data  by  the  analysis  function,  as  delivered,  meet  the 
requirements  of  the  CCAWS  and  AGMS  Project  Managers.  These 
calculations,  however,  were  designed  from  within  the  Microsoft 
Access  database  program  and  can  be  altered  to  perform 
additional  analyses,  as  necessary.  The  analyses  currently 
performed  by  the  analysis  function  are: 
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•  The  percentage  of  actual  responses  of  each 
functional  category,  by  organization  and  year,  to 
the  maximum  possible  response. 

•  The  average  response  for  each  assessment  item,  by 
year,  organization,  office,  and  demographic 
category. 

•  Groups  qualitative  responses  by  year,  organization, 
office,  and  demographic  category. 

These  calculations  are  then  compared  to  and  against  each  other 

to  provide  the  manager  with  an  in-depth  analysis  of  each  item 

measured  by  the  self-assessment. 

(3)  Presentation.  The  presentation  function  of 
the  data  storage,  analysis,  and  presentation  application  is 
designed  to  graphically  present  analyses  performed  by  the 
analysis  function  in  a  manner  that  provides  useful  feedback 
and  information  to  the  project  office's  management.  At  the 
time  of  testing,  this  function  was  designed  to  print  a 
"canned"  or  fixed  set  of  graphs,  tables,  and  charts  to  be  used 
by  management  personnel.  During  the  demonstration  of  the  data 
storage,  analysis,  and  presentation  application,  however,  the 
CCAWS  Project  Manager  commented  that  the  presentation  function 
would  be  more  useful  if  it  allowed  managers  to  view  these 
graphs,  tables,  and  chart  at  their  workstation  and  print  only 
those  items  that  they  deemed  significant.  The  production 
version  of  the  data  storage,  analysis,  and  presentation 
application  incorporates  this  suggestion,  allowing  each 
presentation  item  to  be  viewed  on  screen  and  printed,  as 
necessary.  The  presentation  graphics  are  structured  in  a 
hierarchical  fashion.  As  the  database  application  is  opened, 
the  manager  is  presented  with  an  overall  assessment  of  the 
organization  depicting  each  year  the  assessment  instrument  has 
been  used.  The  manager  is  provided  a  series  of  push  buttons 
in  the  screens  margins  allowing  him/her  to  choose  the  year 
he/she  desired  to  examine.  Once  a  year  has  been  selected,  the 
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graph  depicting  that  year  by  functional  category  is  presented 
to  the  manager.  This  procedure  is  then  repeated,  each  level 
providing  increased  detail,  until  completed.  Presentation 
graphics  depicting  six  levels  of  detail  are  included  in  the 
application  (i.e.  Item  1.1  by  year,  organization,  office, 
position,  and  demographic  category)  as  delivered. 
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VII.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  draw  conclusions 
concerning  the  effectiveness  of  the  CCAWS/AGMS  Unit  Self- 
assessment  and  the  prototype  software  application  to  measure 
the  program  health  and  customer  satisfaction  of  a  military 
project  management  office,  based  on  the  issues  and  findings 
revealed  in  the  data  analysis  and  discussion.  The  chapter  is 
divided  into  five  sections:  Conclusions  drawn  from  the 
analysis  and  discussion;  recommendations  for  implementing  the 
CCAWS  Unit  Self-assessment  and  the  prototype  software 
application;  thesis  contributions;  research  limitations 
discovered  during  the  data  analysis;  and  recommendations  for 
further  study. 

B.  CONCLUSIONS 

This  study  addresses  the  primary  research  question: 

Does  the  proposed  unit  self-assessment  package  satisfy  the 
sponsors’  requirement  to  efficiently  and  effectively  assess 
and  track  program  health  and  customer  satisfaction? 

The  proposed  unit  self-assessment  package,  consisting  of 
the  CCAWS/AGMS  Unit  Self-assessment  and  the  Unit  Self- 
assessment  software,  version  1,  do  provide  the  military 
project  manager  with  the  tools  necessary  to  efficiently  and 
effectively  assess  the  program  health  and  internal  customer 
satisfaction  of  a  project  management  office.  The  CCAWS/AGMS 
Unit  Self-assessment  does  not,  however,  satisfy  the  sponsors' 
requirement  to  assess  the  external  customers'  satisfaction 
with  the  products  and  services  provided  by  the  project  office. 

This  thesis  also  sufficiently  addresses  each  subsidiary 
question  listed  below: 
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1 .  Does  the  proposed  assessment  address  and  correct  the 
deficiencies  of  the  Self-assessment  Guide  for  Organ¬ 
izational  Performance  and  Customer  Satisfaction, 
identified  by  the  sponsors'  management  team  and 
users? 

The  CCAWS/AGMS  Unit  Self-assessment  provides  enhanced 
solutions  to  each  of  the  problems  identified  by  the  management 
teams  and  personnel  from  the  CCAWS  and  AGMS  Project  Management 
Offices.  Specifically,  the  CCAWS/AGMS  Unit  Self-assessment 
addresses  improvements  in  the  following  areas: 

a.  Applicability  of  Assessment  Items 

All  of  the  items  encompassed  within  the  CCAWS/AGMS 
Unit  Self-assessment  are  applicable  to  a  military  project 
management  office.  Items  determined  as  questionable  by  CCAWS 
and  AGMS  personnel  during  the  candidate  assessment  evaluation 
were  either  modified  to  fit  the  project  management  environment 
or  discarded. 

b.  Item  Wording  and  Content 

The  items  in  the  CCAWS/AGMS  Unit  Self-assessment  are 
worded  in  a  manner  that  is  both  clear  and  understandable  by 
all  members  of  a  military  project  office.  Questionable  items 
during  the  candidate  assessment  evaluation  in  this  regard  were 
either  simplified  or  reworded  entirely  to  ensure  each 
participants  full  understanding. 

c.  Assessment  Length 

The  CCAWS/AGMS  Unit  Self-assessment's  length  is 
sufficient  to  ensure  a  comprehensive  evaluation  of  the  program 
health  and  internal  customer  satisfaction  of  a  military 
project  management  office,  without  requiring  an  excessive 
amount  of  time  for  each  participant  to  complete.  The  average 
time  for  each  participant  to  complete  the  candidate  assessment 
will  be  34  minutes,  as  compared  to  an  estimated  90  minutes  to 
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complete  the  previous  Self-assessment  Guide  for  Organizational 
Performance  and  Customer  Satisfaction. 

d.  Target  Audience 

The  items  within  CCAWS/AGMS  Unit  Self-assessment  are 
appropriate  to  solicit  an  accurate  response  from  all  personnel 
within  a  military  project  management  office.  Items  in  which 
the  subject  matter  or  organizational  level  restricted  the 
ability  of  all  personnel  to  provide  an  accurate  response, 
based  on  their  personal  knowledge  were  either  reduced  in  scope 
to  enable  all  personnel  to  respond  or  eliminated  from  the 
assessment . 

e.  Qualitative  Feedback 

The  CCAWS/AGMS  Unit  Self-assessment  enables  each 
participant  to  provide  both  quantitative  and  qualitative 
feedback  in  each  evaluated  category,  as  opposed  to  the  Self- 
assessment  Guide  for  Organizational  Performance  and  Customer 
Satisfaction,  which  incorporates  qualitative  feedback  by  means 
of  a  consensus-based  management  approach. 

2.  Is  the  proposed  software  program  an  effective  method 
to  administer  the  proposed  assessment? 

The  independently  developed  prototype  software 
application  is  an  effective  method  of  automating  the  unit 
self-assessment  process  and  administering  the  CCAWS/AGMS  Unit 
Self-assessment.  The  software  has  proven  effective  in 
reducing  the  overall  time  associated  with  the  assessment 
process  by  simplifying  the  activities  of  both  the  assessment 
participants  and  the  designated  unit  facilitators.  The 
average  time  for  each  participant  to  complete  the  candidate 
assessment  using  the  prototype  software  will  be  34  minutes,  as 
compared  to  an  estimated  90  minutes  to  complete  the  Self- 
assessment  Guide  for  Organizational  Performance  and  Customer 
Satisfaction  using  manual  procedures.  This  amounts  to  a 
savings  of  93V3  man-hours,  in  a  project  office  of  100  assigned 
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The  software 


personnel,  per  assessment  iteration, 
application's  analysis  and  presentation  function 
instantaneously  performs  the  calculations  required  to  analyze 
the  collected  data,  simultaneously  formatting  the  analyzed 
data  into  statistically  relevant  charts  and  graphs  for 
presentation  of  the  analysis.  This  procedure,  accomplished 
manually,  is  estimated  to  take  one  person  a  full  eight  hour 
work  day  to  complete.  Thus,  the  prototype  software 
application  is  estimated  to  reduce  the  overall  assessment 
process  time  by  506%  man-hours,  over  a  five  year  period. 

The  analysis  of  the  prototype  software  application 
also  addresses  the  following  sponsor  automation  requirements: 

a.  Compatibility  with  Existing  Computer  Resources 

The  prototype  software  application  is  fully 

compatible  with  the  current  hardware  and  software 
configurations  within  the  CCAWS  and  AGMS  Project  Offices.  The 
minimum  system  specifications  of  the  LANs  and  computer 
workstations  in  each  of  the  project  offices  are  sufficient  to 
enable  the  installation  and  operation  of  the  prototype 
software  application.  Programs  within  the  prototype 

application  are  specifically  designed  to  operate  from  within, 
or  interact  with,  Microsoft's  Access  database  program,  which 
is  used  by  both  the  CCAWS  and  AGMS  Project  Offices. 

b.  User  Friendly 

Based  on  the  responses  from  the  CCAWS  and  AGMS 
participants  in  the  prototype  software  testing,  this 
researcher  concludes  that  the  application  is  "user  friendly". 
During  this  testing,  none  of  the  participants  encountered  any 
significant  difficulties  related  to  the  use  of  the  software 
application.  Most  of  the  participants  stated  that  the 
software  environment,  and  many  of  the  objects  and  operators 
were  already  familiar  to  them.  This  is  credited  to  the  fact 
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that  the  software  incorporates  the  use  of  the  Microsoft 
Windows  environment,  used  in  existing  applications  within  both 
the  project  offices.  For  those  objects  that  are  unique  to  the 
prototype  application,  participants  of  the  prototype  testing 
were  observed  during  the  evaluation  process  to  determine  their 
initial  reactions  to  these  items.  The  data  collected  from 
these  observations  were  then  used  to  adjust  the  initial 
prototype,  aligning  the  commands  activating  the  application's 
functions  to  correspond  with  the  participants'  inclinations  to 
activate  them,  thus  further  enhancing  the  "user  friendliness" 
of  the  application. 

c.  Accessibility 

The  CCAWS/AGMS  Unit  Self-assessment  is.  easily 
accessed  by  all  members  of  the  CCAWS  and  AGMS  project  offices. 
The  assessment  software,  once  installed,  is  located  on  the 
organization's  LAN.  Each  workstation  within  the  organization 
connected  to  this  network  enables  the  assessment  participants 
unrestricted  access  to  the  application' s  user  interface 
function. 

3.  Does  the  proposed  unit  self-assessment  package 

provide  project  managers  with  the  information  and 
tools  necessary  to  track  and  improve  program  health 
and  customer  satisfaction? 

The  recommended  self-assessment  package,  consisting  of 
the  CCAWS/AGMS  Unit  Self-assessment  and  the  prototype  software 
application  provides  the  project  office's  management  personnel 
with  sufficient  information  and  adequate  tools  to  evaluate, 
monitor,  and  improve  the  program  health  and  internal  customer 
satisfaction  of  the  project  office.  The  analysis  of  feedback 
collected  during  the  candidate  assessment  evaluation  and  the 
prototype  software  testing  concerning  the  overall  assessment 
package  addresses  the  following  items: 
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a.  Data  Analysis 

The  types  and  depth  of  the  analyses  incorporated  by 
the  unit  self-assessment  package  are  sufficient  to  meet  the 
sponsors'  requirements  to  generate  statistically  relevant  data 
that  provide  meaningful  feedback  concerning  the  status  of  the 
program  health  and  internal  customer  satisfaction  of  the 
project  office. 

b.  Data  Presentation 

The  unit  self-assessment  package  incorporates  the 
use  of  computer  generated  charts  and  graphs  to  present  the 
results  of  generated  feedback  and  analyses  in  a  manner  that  is 
clear,  concise,  and  useful  to  the  project  office's  management 
team.  These  charts  and  graphs  can  be  either  viewed  on  screen 
from  any  available  workstation  connected  to  the  organization's 
LAN  or  printed  to  support  additional  analyses,  discussion,  or 
other  purposed  as  desired  by  management. 

c .  Data  Storage 

The  unit  self-assessment  package  exceeds  the 
sponsors'  requirement  to  store  collected  data  for  a  period  of 
not  less  than  five  years.  The  assessment  application  is 
capable  of  storing  data  collected  by  the  assessment  process 
for  an  unlimited  period.  If,  however,  data  are  accumulated 
for  a  period  greater  than  the  five  year  requirement,  this 
retention  of  excess  data  may  cause  the  presentation  of 
analyses  to  become  confused  and  cluttered  and  may  also  result 
in  a  noticeable  decrease  in  the  speed  at  which  these  charts 
and  graphs  are  displayed. 

C .  RECOMMENDATIONS 

Efficiency  in  the  acquisition  of  major  defense  systems  is 
of  paramount  importance,  given  today' s  environment  of 
declining  budgets  and  Congressional  and  public  skepticism. 
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To  achieve  this  efficiency,  the  project  office,  as  the 
responsible  agency  for  such  acquisitions,  must  in  turn  perform 
its  designated  functions  both  effectively  and  efficiently. 
This  can  only  be  accomplished  by  establishing  systems  within 
the  program  structure  to  assess,  monitor,  and  improve  the 
critical  factors  that  determine  the  efficiency  and  the 
effective  operation  of  the  project  office.  The  two  most 
important  indicators  reflecting  the  efficiency  and 
effectiveness  of  the  project  office  are  program  health  and 
customer  satisfaction.  To  this  end,  this  study  recommends  the 
following  items: 

1 .  Conduct  Periodic  Assessments  Using  the  CCAWS/AGMS 
Unit  Self-assessment  Package 

The  CCAWS/AGMS  Unit  Self-assessment  package,  consisting 
of  the  CCAWS/AGMS  Unit  Self-assessment  and  the  prototype 
software  application,  is  both  an  effective  and  efficient 
method  to  measure  the  program  health  and  internal  customer 
satisfaction  of  a  project  management  office.  Assessments 
should  be  conducted  semi-annually  using  this  automated 
assessment  instrument  to  identify  and  monitor  areas  needing 
improvement  with  respect  to  the  critical  factors  that 
determine  the  program  health  and  internal  customer 
satisfaction  of  the  project  office. 

2.  Conduct  Periodic  Reviews  of  the  CCAWS/AGMS  Unit  Self- 
assessment  Functional  Areas  and  Items 

From  time  to  time  occurrences  may  arise  in  which  the 
strategy,  goals,  objectives,  priorities,  or  functions  of  the 
project  office  may  change  due  to  external  influences  or  as  the 
result  of  internal  initiatives.  To  ensure  the  accurate 
measurement  of  the  critical  factors  that  determine  the  program 
health  and  internal  customer  satisfaction  of  the  project 
office,  the  assessment  instrument  used  to  perform  this 
measurement  should  be  modified  to  reflect  these  changes,  as 
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necessary.  To  this  end,  the  CCAWS/AGMS  Unit  Self-assessment 
should  be  reviewed  annually  to  ensure  critical  changes  in  the 
functioning  of  the  project  office  are  reflected  in  a  timely 
manner . 

3.  Institute  a  Separate  Assessment  Focusing  on  Customer 
Satisfaction 

As  stated  earlier,  one  of  the  two  most  important 
indicators  of  the  efficient  and  effective  functioning  of  the 
project  management  office  is  customer  satisfaction.  The 
CCAWS/AGMS  Unit  Self-assessment,  however,  measures  only  the 
customer  satisfaction  internal  to  the  project  office  and  does 
not  address  the  level  of  satisfaction  experienced  by  the 
project  office's  external  customers.  Given  the  importance  of 
this  factor,  an  additional  assessment  instrument  should  be 
designed  for  the  purpose  of  measuring  this  factor.  This 
assessment,  once  developed,  should  be  conducted  using  the 
existing  prototype  software  application  recommended  by  this 
study. 

D .  CONTRIBUTIONS 

This  thesis  provides  the  project  manager  with  a  viable 
model  depicting  the  critical  organizational  design  factors 
impacting  on  the  program  health  and  internal  customer 
satisfaction  of  the  military  project  office.  This  study  also 
provides  the  project  manager  an  implementing  instrument,  based 
on  the  above  model,  with  which  to  assess  the  level  of  program 
health  and  internal  customer  satisfaction  of  the  project 
office.  Finally,  this  research  effort  has  produced  an 
independent  software  application,  specifically  designed  to 
automate  the  self-assessment  process  within  a  military  project 
management  office  which  will  significantly  reduce  the  man¬ 
hours  associated  with  the  self-assessment  process. 
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E.  RESEARCH  LIMITATIONS 


This  section  describes  the  limitations  concerning  this 
research  effort  surfaced  during  the  candidate  assessment 
evaluation  and  the  prototype  software  application  testing. 
The  items  listed  here  are  in  addition  to  those  previously 
discussed  in  Chapter  I  of  this  thesis. 

1.  Reliability  and  Validity  of  Assessment  Items 

The  CCAWS/AGMS  Unit  Self-assessment  is  a  pilot 
instrument.  Many  of  the  items  used  in  the  assessment  are 
extractions  or  modifications  from  the  USAREUR  Organizational 
Diagnostic  Survey  (modified) .  These  items  have  not  been 
tested  to  determine  their  reliability  or  validity  to  measure 
the  factors  with  which  they  have  been  grouped.  They  are, 
however,  deemed  to  have  face  validity,  based  on  responses  from 
various  personnel  with  longevity  in  the  project  management 
discipline.  Additionally,  items  which  are  included  in  factor 
areas  that  were  added  based  on  feedback  acquired  during  the 
candidate  assessment  evaluation  have  not  been  evaluated  and 
therefore  cannot  be  said  to  contain  face  validity.  This 
limitation  is  applicable  to  those  items  within  the  customer 
satisfaction,  integrated  product  team,  and  products/services 
functional  areas. 

2.  Software  Compatibility 

The  prototype  software  application  is  designed  to  operate 
effectively  across  the  organization's  LAN.  The  documentation 
supporting  the  development  of  the  prototype  application  has 
confirmed  its  compatibility  with  the  types  and  brands  of  LANs 
used  in  both  the  CCAWS  and  AGMS  project  offices.  A  physical 
test,  however,  to  verify  the  application's  compatibility  with 
the  LANs  was  not  conducted  due  to  time  constraints  and  the 
candidate  assessment  evaluation  and  software  testing  methods 
chosen  by  this  researcher. 
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F.  RECOMMENDATIONS  FOR  FURTHER  STUDY 

1.  CCAWS/AGMS  Unit  Self-assessment  Item  Reliability  and 
Validity  Testing 

The  assessment  instrument  recommended  by  this  study,  the 
CCAWS/AGMS  Unit  Self-assessment,  contains  face  validity  only, 
based  on  the  expert  opinions  of  personnel  with  longevity  in 
the  field  of  project  management,  and  has  not  been  subjected  to 
any  form  of  psychometric  testing  or  evaluation.  A  study  of 
the  reliability  and  validity  of  the  items  incorporated  in  the 
CCAWS/AGMS  Unit  Self-assessment  will  confirm  or  deny  the 
effectiveness  of  these  items  to  accurately  measure  the  program 
health  and  customer  satisfaction  of  a  project  management 
office . 

2.  Customer  Satisfaction  Survey  Development 

The  CCAWS/AGMS  Unit  Self-assessment  provides  a  means  with 
which  to  measure  the  program  health  and  internal  customer 
satisfaction  of  a  project  management  office.  This  instrument 
does  not,  however,  address  the  sponsors'  requirement  to 
measure  the  external  customers'  level  of  satisfaction  with  the 
products  and  services  delivered  by  the  project  office. 
Research  in  this  area  would  result  in  the  development  of  an 
assessment  instrument  designed  for  use  by  the  project 
management  office  to  measure  the  satisfaction  of  its  external 
customers . 
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APPENDIX  A:  THE  USAREUR  ORGANIZATIONAL  DIAGNOSTIC  SURVEY 
(MODIFIED) 

Group:  1.  Strategy  (Mission  -  Goals  -  Objectives) 

1.1  -  To  what  extent  do  you  understand  the  mission  of 

the  Project  Office?  (Scale  1) 

1.2  -  To  what  extent  do  you  understand  the  objectives 

of  the  Project  Office?  (Scale  1) 

1.3  -  To  what  extent  are  the  objectives  of  the  Project 

Office  realistic?  (Scale  1) 

1.4  -  To  what  extent  do  the  objectives  of  the  Project 

Office  support  the  goals  of  Program  Executive 
Office?  (Scale  1) 

1.5  -  To  what  extent  are  the  people  in  the  Project 

Office  committed  to  achieving  its  objectives? 

(Scale  1) 

1.6  -  I  am  satisfied  with  my  role  in  setting 

organizational  objectives.  (Scale  2) 

1.7  -  To  what  extent  are  the  Project  Office's 

objectives  for  mission  accomplishment  consistent 
with  the  demands  of  its  environment?  (Scale  1) 

1.8  -  To  what  extent  are  the  Project  Office's 

objectives  for  mission  accomplishment  consistent 
with  the  availability  of  its  resources? 

(Scale  1) 

1.9  -  To  what  extent  does  the  work  actually  performed 

support  the  Project  Office's  stated  objectives? 

(Scale  1) 

1.10  -  The  strategy  of  the  Project  Office  could  be 

improved  by:  (open) 

Group :  2 .  Task 

2.1-1  know  what  tasks  I  am  responsible  for 
accomplishing.  (Scale  2) 
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2.2  -  To  what  extent  do  your  assigned  tasks  include 

work  that  you  consider  to  be  outside  your  area  of 
responsibility?  (Scale  1) 

2.3  -  To  what  extent  are  your  assigned  task  deadlines 

reasonable?  (Scale  1) 

2.4  -  To  what  extent  does  the  accomplishment  of  your 

duties  depend  on  the  coordinated  input  of  other 
people?  (Scale  1) 

2.5  -  To  what  extent  are  you  able  to  plan  your 

activities  for  the  following  week?  (Scale  1) 

2.6  -  To  what  extent  do  your  activities  occur  as  you 

have  them  planned?  (Scale  1) 

2.7-1  have  sufficient  authority  to  accomplish  my 
assigned  tasks.  (Scale  2) 

2.8  -  To  what  extent  do  the  Project  Office's  procedures 

hinder  task  accomplishment?  (Scale  1) 

2.9  -  Task  accomplishment  in  the  Project  Office  could 

be  improved  by:  (open) 


Group:  3.  Structure 


3.1  -  To  what  extent  does  the  Project  Office's  present 

structure  (the  way  people  are  divided  up  to  do 
work)  facilitate  work?  (Scale  1) 

3.2  -  To  what  extent  does  the  Project  Office's  present 

structure  facilitate  the  flow  of  information? 

(Scale  1) 

3.3  -  To  what  extent  does  the  Project  Office's  present 

structure  facilitate  decision  making? 

(Scale  1) 

3.4  -  Our  informal  work  structure  complements  the 

established  formal  structure.  (Scale  2) 

3.5  -  To  what  extent  are  individual  responsibilities 

defined  in  the  Project  Office?  (Scale  1) 

3.6  -  To  what  extent  is  the  accountability  for  final 

products  clear?  (Scale  1) 
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3.7 

3.8 

3.9 

3.10 

3.11 

3.12 

Group :  4 . 

4.1 

4.2 

4.3 

4.4 

4.5 

4.6 


-  To  what  extent  are  people  who  are  responsible  for 
task  accomplishment  provided  the  necessary 
resources?  (Scale  1) 

-  To  what  extent  do  people  who  are  responsible  for 
task  accomplishment  have  the  necessary  authority? 

(Scale  1) 

-  To  what  extent  are  there  barriers  to  the  lateral 
flow  of  information  in  the  Project  Office? 

(Scale  1) 

-  To  what  extent  is  necessary  information  passed 
up  and  down  the  chain  of  command?  (Scale  1) 

-  To  what  extent  do  the  policies  of  the  Project 
Office  hinder  mission  accomplishment?  (Scale  1) 

-  The  way  we  are  divided  up  to  do  things  and  the 
procedures  we  follow  could  be  improved  by: 

(open) 

People 

-  The  people  in  the  Project  Office  have  the  skills 
and  knowledge  necessary  to  accomplish  their 
assigned  tasks.  (Scale  2) 

-  The  people  in  the  Project  Office  clearly  express 
their  knowledge  of  and  support  for  the  Project 
Office's  objectives.  (Scale  2) 

-  Personnel  in  the  Project  Office  understand  their 
jobs  and  how  they  contribute  to  the  overall 
organizational  effort.  (Scale  2) 

-  There  is  a  high  degree  of  personal  commitment  to 
the  Project  Office.  (Scale  2) 

-  The  people  in  the  Project  Office  are  warm  and 
friendly  toward  each  other.  (Scale  2) 

-  The  Project  Office  is  seen  as  a  good  place  to 
work  because  its  people  are  treated  right. 

(Scale  2) 
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4.7 


-  To  what  extent  does  the  structure  of  the  Project 
Office  help  people  as  they  do  their  work. 

(Scale  1) 

4.8  -  The  following  things  need  to  be  done  in  order  to 
improve  the  fit  between  the  kind  of  people  we  now 
have  and  the  kind  we  ought  to  have:  (open) 


Group:  5.  Rewards 


5.1  -  To  what  extent  are  rewards  in  the  Project  Office 

contingent  upon  results  rather  than  tenure  or 
rank?  (Scale  1) 

5.2  -  The  criteria  for  receiving  rewards  are  clear  to 

me.  (Scale  2) 

5.3  -  To  what  extent  do  you  understand  the  performance 

standards  of  the  Project  Office?  (Scale  1) 

5.4  -  To  what  extent  do  supervisors  in  the  Project 

Office  inform  their  subordinates  of  their  duty 
performance?  (Scale  1) 

5.5  -  To  what  extent  are  you  satisfied  with  the  award 

system  in  the  Project  Office.  (Scale  1) 

5.6  -  To  what  extent  does  the  reward  system  in  the 

Project  Office  motivate  people  to  achieve  better 
results.  (Scale  1) 

5.7  -  The  reward  system  in  the  Project  Office  could  be 

improved  by:  (open) 


Group:  6.  Processes 

6.1  -  Communication  is  good  throughout  the  Project 

Office,  (i.e.  the  people  know  what  is  going  on). 

(Scale  2) 

6.2  -  Coordination  is  good  throughout  the  Project 

Office.  (Scale  2) 

6.3  -  To  what  extent  is  conflict  dealt  with  openly  and 

managed  constructively  in  the  Project  Office? 

(Scale  1) 
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6.4  -  To  what  extent  are  clear  individual  objectives 

and  performance  criteria  established  for  each 
person  in  the  Project  Office?  (Scale  1) 

6.5  -  Decisions  are  made  at  the  most  appropriate  level 

in  the  Project  Office.  (Scale  2) 

6.6  -  To  what  extent  do  the  formally  established 

policies  and  procedures  in  the  Project  Office 
match  what  people  actually  do?  (Scale  1) 

6.7  -  To  what  extent  do  people  in  the  Project  Office 

understand  which  decisions  they  can  make 
themselves  and  which  ones  must  involve  other 
people?  (Scale  1) 

6.8  -  The  informal  power  structure  of  the  Project 

Office  complements  the  formal  structure. 

(Scale  2) 

6.9  -  The  processes  of  the  Project  Office 

(communication,  decision  making,  conflict 
management,  individual  role  clarification)  could 
be  improved  by:  (open) 

Group :  7 .  Leadership 

7.1  -  To  what  extent  does  the  leadership  of  the  Project 

Office  provide  clear  guidance  on  tasks  that  must 
be  accomplished?  (Scale  1) 

7.2  -  To  what  extent  is  the  leadership  of  the  Project 

Office  proactive  (focused  on  the  future)  rather 
than  reactive?  (Scale  1) 

7.3  -  The  leadership  of  the  Project  Office  demonstrates 

that  they  trust  me  to  do  my  job  well.  (Scale  2) 

7.4  -  The  informal  leaders  in  the  Project  Office 

complement  the  formal  leadership.  (Scale  2) 

7.5  -  To  what  extent  is  the  leadership  of  the  Project 

Office  respected  by  the  people  in  the  Project 
Office?  (Scale  1) 

7.6  -  To  what  extent  are  the  leaders  of  the  Project 

Office  aware  of  what  is  going  on  "in  the 
trenches"  of  the  Project  Office?  (Scale  1) 
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7.7  -  The  leaders  in  the  Project  Office  are  genuinely 

people  oriented.  (Scale  2) 

7.8  -  The  leadership  of  the  Project  Office  could  be 

improved  by:  (open) 


Group  8 .  Technology 


8.1  -  To  what  extent  is  the  equipment  of  the  Project 

Office  (computers,  word  processors,  copiers, 
etc.)  sufficient  to  accomplish  its  mission? 

(Scale  1) 

8.2  -  To  what  extent  does  the  Project  Office's  budget 

reflect  the  funds  necessary  to  purchase  needed 
equipment?  (Scale  1) 

8.3  -  The  Project  Office  makes  full  use  of  the 

equipment  it  does  have.  (Scale  2) 

8.4  -  To  what  extent  are  personnel  in  the  Project 

Office  trained  in  the  proper  use  and  maintenance 
of  their  equipment?  (Scale  1) 

8.5  -  To  what  extent  is  routine,  repetitive  work  in  the 

Project  Office  automated?  (Scale  1) 

8.6  -  Our  management  information  system  (MIS)  supports 

the  Project  Office  rather  than  vice  versa. 

(Scale  2) 

8.7  -  To  what  extent  is  additional  equipment  and  timely 

external  technical  support  available?  (Scale  1) 

8.8  -  The  technical  support  of  the  Project  Office  can 

be  improved  by:  (open) 


Group :  9 .  General 

9.1  -  The  things  I  like  most  about  the  Project  Office 

are:  (open) 

9.2  -  The  things  I  like  least  about  the  Project  Office 

are: (open) 

9.3  -  If  I  were  in  charge  of  the  Project  Office,  I 

would:  (open) 
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Response  Scales 


Scale  1 

Value 

Scale  2 

Totally 

«*  6  ► 

Strongly  Agree 

Substantially 

<  5  ► 

Agree 

Reasonably 

m 

Slightly  Agree 

Moderately 

«  3  ► 

Slightly  Disagree 

Barely 

mam 

Disagree 

None 

<  1  ► 

Strongly  Disagree 

Don ' t  Know 

<  0  ► 

Don ' t  Know 
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APPENDIX  B:  THE  USAREUR  ORGANIZATIONAL  DIAGNOSTIC  SURVEY 


Group:  1.  Strategy  (Mission  -  Goals  -  Objectives) 

1.1  -  I  understand  the  strategy  (mission,  goals, 

objectives) of  this  organization. 

1.2  -  The  organization  has  a  set  of  realistic 

objectives  which  support  the  goals  of  the  next 
higher  organization. 

1.3  -  This  organization's  strategy  will  project  the 

organization  into  the  future  in  the  most 
effective  manner. 

1.4  -  There  is  a  high  degree  of  commitment  to  this 

organization's  mission  and  objectives. 

1.5  -  I  am  satisfied  with  my  role  in  setting 

organizational  objectives. 

1.6  -  This  organization's  strategy  for  mission 

accomplishment  (link  between  mission,  goals, 
objectives)  is  consistent  with  the  demands  of  its 
environment  and  resources. 

1.7  -  The  work  actually  done  in  this  organization 

matches  that  dictated  by  its  strategy. 

1.8  -  The  strategy  of  this  organization  could  be 

improved  by: 

Group :  2 .  Task 

2.1-1  know  what  tasks  I  am  responsible  for 
accomplishing. 

2.2  -  Most  of  the  tasks  I  am  assigned  fall  within  what 

I  consider  my  area  of  responsibility. 

2.3  -  The  suspense  dates  associated  with  most  of  my 

assigned  tasks  are  reasonable. 

2.4  -  My  task  accomplishment  is  usually  dependent  upon 

input  from  other  people. 

2.5  -  I  can  usually  plan  ahead  what  my  tasks  for  the 

following  day  will  be. 
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2.6 

2.7 

2.8 

Group :  3 

3.1 

3.2 

3.3 

3.4 

3.5 

3.6 

3.7 

3.8 

Group :  4 

4.1 

4.2 


-  I  have  enough  authority  to  accomplish  my  assigned 
tasks . 

-  Our  procedures  and  management  practices 
facilitate  task  accomplishment. 

-  Task  accomplishment  in  this  organization  could  be 
improved  by: 

Structure 

-  Our  present  structure  (the  way  we  are  divided  up 
to  do  work  —  the  wiring  diagram)  facilitates 
work,  information  flow,  and  decision  making. 

-  Our  formal  work  structure  is  the  same  as  our 
informal  structure. 

-  There  is  a  clearly  understood  definition  of  who 
does  what  within  this  organization. 

-  Accountability  for  final  products  is  clear. 

-  People  who  are  responsible  for  task 
accomplishment  have  the  necessary  resources  and 
authority. 

-  Information  flows  easily  across  boundaries  in 
this  organization. 

-  Our  rules  and  policies  help  us  get  the  job  done. 

-  The  way  we  are  divided  up  to  do  things  and  the 
procedures  we  follow  to  do  them  could  be  improved 
by: 

People 

-  The  personnel  in  this  organization  have  the 
skills  and  knowledge  necessary  to  accomplish 
their  assigned  tasks. 

-  The  people  in  this  organization  clearly  express 
their  knowledge  of  and  support  for  the 
organizational  strategy. 


108 


4.3 


4.4  - 

4.5  - 

4.6  - 

4.7  - 

4.8  - 

Group :  5 . 

5.1  - 

5.2  - 

5.3  - 

5.4  - 

5.5  - 

5.6  - 

5.7  - 

5.8  - 


Personnel  in  this  organization  understand  their 
jobs  and  how  they  contribute  to  the  overall 
organizational  effort. 

There  is  a  high  degree  of  personal  commitment  to 
this  organization. 

Organizational  members  are  warm  and  friendly 
toward  each  other. 

This  organization  is  seen  as  a  good  place  to  work 
because  it  treats  its  people  right. 

The  structure/process  arrangements  in  this 
organization  help  people  as  they  do  their  work. 

The  following  things  need  to  be  done  in  order  to 
improve  the  fit  between  the  kind  of  people  we  now 
have  and  the  kind  we  ought  to  have: 

Rewards 

Rewards  in  this  organization  are  contingent  upon 
results  rather  than  tenure  or  rank. 

The  criteria  for  receiving  rewards  are  clear  to 
me . 


Performance  standards  for  individuals  in  this 
organization  are  clear  to  all  concerned. 

It  is  easy  to  find  out  how  you  are  doing  in  this 
organization. 

I  am  satisfied  with  the  award  system  in  this 
organization. 

The  reward  system  supports  the  people  in  their 
desires . 

The  present  reward  system  motivates  people  to 
achieve  better  results. 

The  reward  system  in  this  organization  could  be 
improved  by: 
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Group :  6 . 


Processes 


6.1  -  This  organization  has  good  lateral  communication 

(i.e.  the  people  know  what  is  going  on). 

6.2  -  Conflict  is  dealt  with  openly  and  is  managed 

constructively  in  this  organization. 

6.3  -  Clear  individual  objectives  and  performance 

criteria  are  established  for  each  position  in 
this  organization. 

6.4  -  Decisions  are  made  at  the  most  appropriate  level 

in  this  organization. 

6.5  -  Our  formally  stated  methods  and  practices  match 

what  people  actually  do. 

6.6  -  People  in  this  organization  understand  which 

decisions  they  can  make  and  which  ones  must 
involve  others . 

6.7  -  The  formal  power  structure  is  the  same  as  the 

informal  power  structure. 

6.8  -  The  processes  of  communication,  decision  making, 

conflict  management,  and  individual  role 
clarification  could  be  improved  by: 


Group:  7.  Leadership 


7.1  -  The  leadership  of  this  organization  provides  us 

with  sufficient  clear  guidance  on  tasks  we  must 
accomplish. 

7.2  -  The  leadership  of  this  organization  focuses  on 

the  future  (proactive)  rather  than  the  past 
(reactive) . 

7.3  -  The  leadership  of  this  organization  demonstrates 

that  it  trusts  me  to  do  my  job  well. 

7.4  -  The  "formal"  leaders  of  this  organization  are 

also  the  "informal". 

7.5  -  The  leadership  of  this  organization  is  respected 

by  the  organization. 
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7.6  -  The  leadership  of  this  organization  is  aware  of 

what  is  going  on  "in  the  trenches". 

7.7  -  The  leaders  in  this  organization  are  genuinely 

people  oriented. 

7.8  -  The  leadership  of  this  organization  could  be 

improved  by: 


Group:  8.  Technology 


8.1  -  This  organization  has  sufficient  equipment 

(computers,  word  processors,  calculators,  etc.) 

To  accomplish  its  mission. 

8.2  -  The  money  to  purchase  new  equipment  that  we  do 

need  but  do  not  have  is  reflected  in  our  budget. 

8.3  -  This  organization  makes  full  use  of  the  equipment 

it  does  have. 

8.4  -  The  personnel  in  this  office  have  sufficient 

knowledge  to  operate  and  maintain  the  equipment 
it  does  have. 

8.5  -  Most  of  our  routine,  repetitive  work  is 

automated. 

8.6  -  Our  management  information  system  (MIS)  supports 

our  organization  rather  than  vice  versa. 

8.7  -  If  this  organization  does  not  have  the 

technological  support  needed,  it  is  available 
from  another  organization  on  a  timely  basis. 

8.8  -  The  technological  support  and  equipment  we  need 

to  do  a  better  job  fall  into  these  areas: 


Group :  9 .  General 

9.1  -  The  thing  I  like  most  about  this  organization  is: 

9.2  -  The  thing  I  like  least  about  this  organization 

is : 

9.3  -  If  I  were  in  charge  of  this  organization,  I 

would: 
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APPENDIX  C:  THE  CCAWS/AGMS  UNIT  SELF-ASSESSMENT 


Group:  1.  Mission/Objectives 


1.1  -  To  what  extent  do  you  understand  the  mission  of 
the  Project  Office?  (Scale  1) 


1.2  -  To  what  extent  do  you  understand  the  objectives 

of  your  office?  (Scale  1) 

1.3  -  I  agree  with/support  the  objectives  of  my  office? 

(Scale  2) 


1.4  -  To  what  extent  does  the  work  actually  performed 
support  your  office's  stated  objectives? 

(Scale  1) 


Group:  2.  Task  Requirements 


2.1-1  know  what  tasks  I  am  responsible  for 
accomplishing.  (Scale  2) 

2.2  -  To  what  extent  are  your  assigned  task  deadlines 

reasonable?  (Scale  1) 

2.3  -  To  what  extent  does  the  accomplishment  of  your 

duties  depend  on  input  from  other  people? 

(Scale  1) 

2.4  -  To  what  extent  are  you  able  to  plan  your 

activities  for  the  following  week?  (Scale  1) 

2.5  -  To  what  extent  do  your  activities  occur  as  you 

have  them  planned?  (Scale  1) 

2.6  -  I  have  sufficient  authority  to  accomplish  my 

assigned  tasks.  (Scale  2) 

2.7  -  To  what  extent  do  the  procedures  in  your  office 

hinder  productivity?  (Scale  1) 

2.8  -  Task  accomplishment  in  my  office  could  be 
improved  by:  (open) 
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Group :  3 . 


Structure 


3.1  -  To  what  extent  does  the  your  office's  present 

structure  (the  way  people  are  divided  up  to  do 
work)  facilitate  task  accomplishment?  (Scale  1) 

3.2  -  To  what  extent  does  the  Project  Office's  present 

structure  facilitate  the  flow  of  information? 

(Scale  1) 

3.3  -  The  informal  work  structure  in  our  office 

complements  the  established  formal  structure. 

(Scale  2) 

3.4  -  To  what  extent  are  your  individual 

responsibilities  defined?  (Scale  1) 

3.5  -  To  what  extent  is  the  accountability  for  final 

products  clear?  (Scale  1) 

3.6  -  To  what  extent  are  you  provided  the  necessary 

resources  to  accomplish  the  tasks  for  which  you 
are  responsible?  (Scale  1) 

3.7  -  I  have  the  necessary  authority  to  accomplish  the 

tasks  for  which  I  am  responsible?  (Scale  2) 

3.8  -  To  what  extent  are  there  barriers  to  the  lateral 

flow  of  information  in  the  Project  Office? 

(Scale  1) 

3.9  -  To  what  extent  do  you  receive  the  information 

necessary  to  perform  your  job?  (Scale  1) 

3.10  -  To  what  extent  do  the  policies  in  your  office 

hinder  mission  accomplishment?  (Scale  1) 

3.11  -  The  way  we  are  divided  up  to  do  things  and  the 

procedures  we  follow  could  be  improved  by: 

(open) 

Group:  4.  People 

4.1-1  have  been  adequately  trained  in  the  skills 
and  knowledge  necessary  to  accomplish  my 
assigned  tasks.  (Scale  2) 
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4.2 


4.3  - 

4.4  - 

4.5  - 


4.6  - 


4.7  - 


Group :  5 . 

5.1  - 

5.2  - 

5.3  - 

5.4  - 


5.5  - 


5.7  - 


The  people  in  my  office  clearly  express 
their  knowledge  of  and  support  for  the  office's 
objectives.  (Scale  2) 

I  understand  how  my  work  contributes  to  the 
overall  organizational  effort.  (Scale  2) 

The  people  in  our  office  are  warm  and 
friendly  toward  each  other.  (Scale  2) 

Our  office  is  seen  as  a  good  place  to 
work  because  its  people  are  treated  well. 

(Scale  2) 

To  what  extent  does  the  structure  of  your 
office  help  people  as  they  do  their  work. 

(Scale  1) 

The  following  things  need  to  be  done  in  order  to 
improve  the  fit  between  the  kind  of  people  we  now 
have  and  the  kind  we  ought  to  have :  (open) 

Integrated  Product  Teams  (IPTs) 

I  am  a  member  of  a  product  oriented  IPT. 

(Scale  2) 

Conflicts  are  openly  resolved  between  members  of 

the  IPT.  (Scale  2) 

To  what  extent  do  you  understand  the  objectives 
of  your  IPT?  (Scale  1) 

To  what  extent  does  your  IPT  include  members 
representing  all  of  the  functional  areas  critical 
to  the  product's  success?  (Scale  1) 

To  what  extent  do  you  feel  the  members  of 
the  IPT  are  focused  on  common  objectives? 

(Scale  1) 

To  what  extent  are  members  of  the  IPT  held 
accountable  for  areas  in  which  they  are 
responsible?  (Scale  1) 

I  understand  how  my  participation  in  the  IPT 
relates  to  the  accomplishment  of  its  objectives. 

(Scale  2) 
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5.8  -  The  effectiveness  of  the  IPT  could  be  improved 

by:  (open) 


Group :  6 . 

6.1 

6.2 

6.3 

6.4 

6.5 

6 . 6 

6.7 

6.8 

6 . 9 

6.10 

6.11 


PIT0C0SS0S 

Communication  is  good  in  our  office,  (i.e.  the 
people  know  what  is  going  on) .  (Scale  2) 

Coordination  is  good  between  offices  within  the 
Project  Office.  (Scale  2) 

To  what  extent  are  personal  conflicts  dealt  with 
openly  in  your  office? 

(Scale  1) 

To  what  extent  are  clear  individual  objectives 
established  for  each  person  in  your  office? 

(Scale  1) 

To  what  extent  are  personal  conflicts  managed 
constructively  in  your  office? 

(Scale  1) 

To  what  extent  are  performance  criteria 
established  for  each  person  in  your  office? 

(Scale  1) 

Decisions  are  made  at  the  most  appropriate  level 
in  my  office.  (Scale  2) 

To  what  extent  do  the  formally  established 
policies  and  procedures  in  your  office  match  what 
people  actually  do?  (Scale  1) 

To  what  extent  do  you  understand  which  decisions 
you  can  make  yourself  and  which  ones  must  involve 
other  people?  (Scale  1) 

-  The  informal  power  structure  within  our  office 
complements  the  formal  structure.  (Scale  2) 

-  The  processes  use  in  our  office  (communication, 
decision  making,  conflict  management,  individual 
role  clarification)  could  be  improved  by:  (open) 
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Group :  7 .  Leadership 

7.1  -  To  what  extent  does  your  immediate  supervisor 

provide  clear  guidance  on  tasks  that  must 
be  accomplished?  (Scale  1) 

7.2  -  To  what  extent  are  the  leaders  in  your  office 

proactive  (focused  on  the  future)  rather  than 
reactive?  (Scale  1) 

7.3  -  My  immediate  supervisor  demonstrates  that  he/she 

trusts  me  to  do  my  job  well.  (Scale  2) 

7.4  -  The  informal  leaders  in  my  office  complement  the 

formal  leadership.  (Scale  2) 

7.5  -  To  what  extent  do  you  respect  the  leaders  in  your 

office?  (Scale  1) 

7.6  -  To  what  extent  is  your  immediate  supervisor  aware 

of  what  is  going  on  "in  the  trenches"  of  your 
Office?  (Scale  1) 

7.7  -  My  immediate  supervisor  is  genuinely  people 

oriented.  (Scale  2) 

7.8  -  The  leadership  in  our  office  could  be  more 

effective  by:  (open) 


Group  8 .  Technology 

8.1  -  To  what  extent  is  the  equipment  in  your  office 

(computers,  word  processors,  copiers,  etc.) 
sufficient  to  accomplish  its  mission? 

(Scale  1) 

8.2  -  Our  office  makes  full  use  of  the  equipment  it 

does  have.  (Scale  2) 

8.3  -  To  what  extent  does  the  basic  approach  to  task 

accomplishment  fit  the  required  tasks  and  the 
resources  available?  (Scale  1) 

8.4  -  To  what  extent  have  you  been  trained  in  the 

proper  use  and  maintenance  of  the  equipment 
you  need  to  perform  your  job?  (Scale  1) 
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8.5  -  To  what  extent  is  routine,  repetitive  work  in 

your  office  automated?  (Scale  1) 

8.6  -  The  Management  Information  System  (MIS)  in  the 

Project  Office  supports  our  office  rather  than 
vice  versa.  (Scale  2) 

8.7  -  The  technology  of  the  Project  Office  can  be 

improved  by:  (open) 

Group  9.  Products /Services 

9.1  -  To  what  extent  are  the  products/services 

delivered  by  your  office  designed  to  meet 
customer  expectations?  (Scale  1) 

9.2  -  Our  office  delivers  professional  products/ 

services  to  its  customers.  (Scale  2) 

9.3  -  To  what  extent  is  your  office  product/service 

oriented?  (Scale  1) 

9.4  -  To  what  extent  do  the  products/services  delivered 

by  your  office  require  rework?  (Scale  1) 

9.5  -  To  what  extent  are  there  procedures  in  your 

office  to  ensure  the  output  of  quality  products 
and  services?  (Scale  1) 

9.6  -  The  products  and  services  delivered  by  our 

office  could  be  improved  by:  (open) 

Group  10.  Customer  Satisfaction 

10.1  -  To  what  extent  are  you  satisfied  with  the 

products  you  receive  from  other  offices  within 
the  PMO?  (Scale  1) 

10.2  -  To  what  extent  are  you  satisfied  with  the 

services  performed  by  other  offices  within  the 
PMO?  (Scale  1) 

10.3  -  To  what  extent  do  the  products/services  you 

receive  from  other  offices  within  the  PMO 
require  rework?  (Scale  1) 
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10.4  -  To  what  extent  does  the  accomplishment  of  your 

assigned  tasks  depend  on  the  products/services 
you  receive  from  other  offices  within  the  PMO? 

(Scale  1) 

10.5  -  In  your  opinion,  which  office  (s)  within  the  PMO 

provides  quality  products/services  to  its 
internal  customers?  (Open) 

10.6  -  In  your  opinion,  which  office (s)  within  the  PMO 

provides  poor  products/services  to  its  internal 
customers?  (Open) 

10.7  -  My  satisfaction  with  the  products/services 

received  from  other  offices  within  the  PMO  could 
be  improved  by:  (open) 

Group:  11.  General 

11.1  -  The  things  I  like  most  about  working  in  my 

office  are:  (open) 

11.2  -  The  things  I  like  least  about  working  in  my 

office  are: (open) 

11.3  -  If  I  were  in  charge  of  my  office,  section, 

branch,  or  division,  I  would:  (open) 


Response  Scales 


Scale  1 

Totally 

Substantially 

Reasonably 

Moderately 

Barely 

None 

Don 1 t  Know 


Scale  2 

Strongly  Agree 
Agree 

Slightly  Agree 
Slightly  Disagree 
Disagree 

Strongly  Disagree 
Don ' t  Know 
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APPENDIX  D:  UNIT  SELF-ASSESSMENT  ITEM  SURVEY 


Respondent  #: 
Organization: 
Position:  _ 


For  each  Item: 
Is  the  wording: 


□ 

1 

□ 
i  _ 

□ 

1 

□ 

1 

□ 

1 

Ambiguous 

□ 

{ 

1 

□ 

......  i 

□ 

. i  . 

□ 

. . .  i  . 

Clear 

□ 

i 

1 

Wordy 

Is  the  content: 

| 

1 

1 

1 

Concise 

□  □  □  □  □ 
l - 1 - 1 - 1 - 1 

Confusing  Understandable 


□  □  □  □  □ 

I - 1 - 1 - 1 - 1 


Irrelevant 

□ 

- 1 - 

□ 

□ 

_ _ I.. . . 

□ 

. . i 

- 1 

Applicable 

□ 

i 

r 

Offensive 

Is  the  Scale: 

□ 

- r 

□ 

i 

i 

□ 

I 

i 

□ 

■ 

Suitable 

□ 

i 

1 

Improper 

I 

i 

i 

. .  1 

Appropriate 

Name : 
Office : 
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APPENDIX  E:  UNIT  ASSESSMENT  AND  SOFTWARE  SURVEY 

Respondent  #:  _  Name:  _ 

Organization:  Office: 

Position:  _ 

A.  Unit  Assessment: 

1.  The  instructions  provided  on  how  to  complete  the 
assessment  are  _ : 


□  □  □  □  □ 


Incomplete  Complete 


2.  The  assessment  itself  is 


Too  Short  About  Right  Too  Long 


Incomplete 


Compr ehens i ve 


3.  _  of  the  assessment  questions  are 

for  a  person  at  my  level  in  the  Project  Office. 


None  Few  Some  Most  All 


Inappropriate 


Appropriate 


4.  What  questions/areas  do  you  feel  are  important 
indicators  of  program  health  that  are  not  included  in  the 
assessment? 


5.  How  long  did  it  take  you  to  complete  the 
assessment? 


_  minutes . 

6.  Please,  provide  any  General  Comments  you  may  have 
about  the  assessment. 


B .  Sof tware : 

1 .  The  instructions  provided  on  how  to  use  the 
software  are: 


□ 

1 

□ 

i 

□ 

1 

□ 

I 

□ 

| 

Confusing 

□ 

1 

1 

□ 

i 

□ 

1 

9 

□ 

1 _ 

Clear 

□ 

| 

Incomplete 

I 

.  1 

Complete 

2.  Is  the  Microsoft  Windows  environment  installed  on 
your  computer  workstation? 

Yes  /  No 
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3.  The  attributes  used  in  the  assessment  software 

(windows,  push  buttons,  drag  and  drop,  etc..)  were  _ 

to  me. 

□  □  □  □  □ 

I - 1 - 1 - 1 - 1 

Unfamiliar  Familiar 


4.  Compared  to  a  "paper  and  pencil"  assessment,  the 
computerized  assessment  is: 


□ 

1 _ 

□ 

□ 

_  i 

□ 

i 

□ 

1 

Less  time 

1 

consuming 

More 

time  consuming 

□ 

1— 

□ 

- 1 - 

□ 

— i — 

□ 

- 1 - 

□ 

- 1 

Easier  to  complete  Harder  to  complete 

5.  Did  you  encounter  any  difficulties  using  the 
software? 


Yes  /  No 

If  yes,  please  describe  the  difficulties. 


6.  What,  if  any,  improvements  to  the  software  would 
you  suggest? 


7.  Please,  provide  any  General  Comments  you  may  have 
about  the  software. 
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APPENDIX  F:  UNIT  SELF-ASSESSMENT  ITEM  SURVEY  DATA 


The  data  collected  from  Survey  #2  concerning  the  effectiveness 
of  the  candidate  self-assessment  items  to  measure  the  program 
health  and  customer  satisfaction  of  a  project  management  office  are 
summarized  in  this  appendix.  The  items  are  listed  as  follows: 


Table 

Functional  Area 

Items 

Table  7 

Strategy 

1.1  -  1.10 

Table  8 

Task 

2.1  -2.9 

Table  9 

Structure 

3.1-3.12 

Table  10 

People 

» 

oo 

Table  11 

Rewards 

5.1  -5.7 

Table  12 

Processes 

6.1  -6.9 

Table  13 

Leadership 

7.1  -7.8 

Table  14 

Technology 

00 

1 

00 

bo 

Table  15 

General 

9.1  -9.3 

Table  6:  Directory  of  Survey  #2  Data  Tables. 
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Clear  1  Concise  1  Ambiguous  |  Relevant  1  Suitable  |  Scale  |  Item  | _ |  Item  |  Clear  |  Concise  |  Ambiguous  |  Relevant  |  Suitable  |  Scale 


szi 


(A69^BJ:q.s)  I  dnojo  -  Asajus  111041  ^usuissassv  -L  ©Tcps! 


Table  8:  Assessment  Item  Survey  Data  -  Group  2  (Task) 
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Clear  1  Concise  |  Ambiguous  |  Relevant  1  Suitable  j  Scale  j  Item  I _ |  Item  I  Clear  I  Concise  I  Ambiguous  I  Relevant  |  Suitable  |  Scale 


Table  10:  Assessment  Item  Survey  Data  -  Group  4  (People) 
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zn 


(spjBMen)  g  dnojQ  -  12412a  A© Ajms  ui©4i  ^ueuissessv  :il  ©iq^I 


Table  12:  Assessment  Item  Survey  Data  -  Group  6  (Processes) 
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Table  14:  Assessment  Item  Survey  Data  -  Group  8  (Technology) 


135 


APPENDIX  G:  QUALITATIVE  RESPONSE  SUMMARY:  UNIT  SELF- 
ASSESSMENT  ITEM  SURVEY 


Item  1.1: 

•  Does  not  apply  to  secretary  position. 


• 

Too  general.  Reduce  scope 

within  the  organization. 

to 

appropriate 

level 

Item 

1.2: 

• 

Too  general.  Reduce  scope 

within  the  organization. 

to 

appropriate 

level 

• 

Change  the  word  "objectives” 

to 

"vision" . 

Item 

1.3: 

• 

Too  general.  Reduce  scope 

within  the  organization. 

to 

appropriate 

level 

• 

Change  the  word  "objectives" 

to 

"vision".  (2) 

• 

Irrelevant  because  these 

are 

fixed  at 

higher 

levels.  Possible  to  ask  about  sub-objectives  that 
support  these  objectives.  (2) 

Item  1.4: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

•  Irrelevant  question,  delete. 

•  A  majority  of  the  people  will  not  know  the  goals  of 

the  PEO.  (3) 

Item  1.5: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

•  Change  the  word  "objectives"  to  "vision”. 

Item  1.6: 

•  Irrelevant  question,  delete. 

•  Not  applicable  to  all  job  positions.  (4) 

Item  1.7: 

•  Change  the  words  "objectives  for  mission 

accomplishment"  to  "vision". 
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•  Environment  and  Demands  are  not  clear.  Question 
may  need  supporting  questions  preceding  it  to  make 
it  clear.  (9) 

•  What  "strategy",  be  specific. 

•  Reword:  ...the  PMO  environment  suitable  to 

accomplish  mission  objectives... 


Item  1.8: 


Irrelevant  question,  delete. 

Reword:  Does  the  PMO  have  the  resources  necessary 
to  accomplish  its  mission? 

Define  "Availability  of  its  resources".  People, 
money,  etc...  (3) 


Item  1. 


9: 


Change  the  word  "objectives"  to  "vision". 

Remove  the  word  "Stated"  from  the  question. 
Possibly  use  goals/objectives. 


Item  1.10: 

•  Change  the  word  "strategy"  to  "strategy/vision" . 

•  Define  "Strategy".  (5) 

•  Reword:  How  could  the  PMO  better  accomplish  its 
mission? 

•  Look  at  asking  what  the  PMO  does  well  also. 

Item  2.2: 

•  Scale:  Reasonable/Moderate  not  clear  for  this 
question.  Confuses  the  user.  Look  for  better 
words.  Suggest  "little". 

•  Delete  question  (could  be  a  grievance) . 

•  Reword:  as  a  statement.  "Tasks  are  assigned".  Use 
scale  #2. 


Item  2.3: 


•  Delete  the  word  "Task". 

•  Wording.  Approach  from  the  viewpoint  of 
unreasonable . 

•  Reword:  Are  your  task  deadlines  reasonable  most  of 
the  time  (or)  how  much  of  the  time?  There  is  a 
certain  amount  of  crisis  management  inherent  in 
this  business. 
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•  Not  specific.  Find  a  way  to  eliminate  crisis 
generated  tasks.  Maybe  use  the  word  normal  or 
routine.  (3) 

Item  2.4: 

•  Reword  as  a  statement.  "Tasks  are  assigned..." 
Use  scale  #2. 

•  Change  the  word  duties  to  tasks  (consistent  with 
group) . 

•  Coordinated  input  is  confusing. 

Item  2.5: 

•  Reword:  "Do  you  plan  your  activities..." 

Item  2.7: 

•  "By  who"  delegated. 

Item  2.9: 

•  Solicit  both  problems  and  solutions,  also  things 
working  well.  (2) 

•  Scale  down  to  branch  office.  Cannot  be  answered 
accurately  by  most  people  in  the  organization  as 
written. 

Item  3.1: 

•  Too  general.  Reduce  scope  to  appropriate  level 
within  the  organization. 

•  Change  the  word  "work"  to  "your  assigned  tasks". 

Item  3.2 : 

•  Type  of  flow  is  confusing  (lateral  or  vertical) . 

Item  3.3: 

•  Who/What  level  of  decision  making  does  this 
question  refer  to?  (2) 

Item  3.4 : 

•  Define  "informal  work  structure".  (2) 

•  Is  this  question  referring  to  the  Integrated 
Product  Teams  or  the  informal  structure? 

•  Change  structure  to  environment.  Change  the  word 
"Formal"  to  "established  PMO  structure". 
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I tem  3.5: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

Item  3.6: 

•  Define  "final  products".  Product  refers  to  the 
final  deliverable  by  the  PMO.  Replace  the  word 
"product".  (2) 

•  Personalize:  Are  you  held  accountable? 

Item  3.7 : 

•  Needs  to  be  more  direct. 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

Item  3.8: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

•  Assumes  all  people  know  the  decision  process. 

Item  3.9: 

•  Define  lateral  (example-  across  functional  areas) . 
(3) 

Item  3.10: 

•  Add  "mission  essential"  or  "important"  information. 

•  Define  "Chain  of  Command". 

Item  3.11: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

Item  3.12: 

•  Not  specific  (how  -  at  what  level) . 

•  Look  at  asking  what  the  PMO  does  well  also. 

Item  4.2: 

•  Clearly  "understand"  and  "support”.  This  should  be 
two  separate  questions. 

Item  4.7: 

•  The  word  structure  is  confusing. 
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Item  4.8: 


•  Look  at  asking  what  the  PMO  does  well  also. 

Item  5.1: 

•  Reword:  drop  the  ending.  Focus  of  the  question 
should  be  on  equity.  Is  the  awards  system 
equitable?  The  use  of  rank  or  tenure  restricts  the 
question.  What  about  sex,  race,  etc...  Use  the 
word  "performance"  instead  of  "results". 

Item  5.3: 

•  Address  individual  performance  standards  instead  of 
overall  office  standards.  (3) 

Item  5.4 : 

•  Suggest  changing  this  question  to  read  "How  many 
times  in  the  past  12  months  has  your  immediate 
supervisor  counseled  you  on  your  duty 
performance?".  (2) 

•  Should  refer  to  the  office,  not  the  PMO.  (4) 

•  Use  the  word  "job"  instead  of  "duty". 

Item  5.5: 

•  Add  "Given  the  current  constraints  on  the  awards 
process",  .  . . 

Item  5.6: 

•  Personalize  to  individual  job.  (Personal  and 

professional  environment  motivate  you  to  achieve.) 

Item  5.7: 

•  Look  at  asking  what  the  PMO  does  well  also. 

Item  6.1: 

•  Specify  "official”  communications. 

Item  6.2: 

•  "Of  What"  (across  functional  areas?)  (Does  it 
achieve  results?)  Possibly  two  separate  questions 
needed. 

Item  6.3: 

•  Define  type  of  conflict  (Mission,  Role,  Personal?) 
(3) 
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Item  6.4: 

•  Delete.  Irrelevant  because  each  individual 

establishes  his/her  own  objectives,  based  on  an 
agreement  between  the  individual  and  his/her 
immediate  supervisor. 

•  Personalize:  ask  about  the  respondent.  (3) 

Item  6.7: 

•  Too  general.  Reduce  scope  to  appropriate  level 
within  the  organization. 

Item  6.8: 

•  Assumes  an  informal  power  structure  exists. 

Item  6.9: 

•  Look  at  asking  what  the  PMO  does  well  also. 

Item  7.1: 

•  Too  general,  needs  to  focus  on  the  office 
leadership,  first  line  supervisor,  not  the  PMO.  (3) 

Item  7.2: 

•  Define  "leadership"  or  reduce  scope.  (2) 

Item  7.3: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

Item  7.4: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

•  Do  the  IPTs  (Integrated  Product  Teams)  complement 
the  chain  of  command? 

•  Assumes  informal  leadership  exists. 

Item  7.5: 

•  Too  general.  Reduce  scope  to  appropriate  level 

within  the  organization. 

Item  7.6: 

•  Too  general .  Reduce  scope  to  appropriate  level 

within  the  organization. 
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Item 

7.7: 

• 

Too  general.  Reduce  scope  to  appropriate  level 
within  the  organization. 

Item 

7.8: 

• 

Too  general.  Reduce  scope  to  appropriate  level 
within  the  organization. 

• 

Look  at  asking  what  the  PMO  does  well  also. 

Item 

8.2: 

• 

Irrelevant  question,  delete. 

Item 

8.3: 

• 

Irrelevant  question,  delete. 

Item 

8.4: 

• 

Irrelevant  question,  delete. 

• 

Define  the  type  of  equipment  "Is  it  talking  about 
the  product  or  the  admin  support  equipment?" 

• 

Possibly  two  questions  being  asked. 

Item 

8.5: 

• 

Irrelevant  question,  delete. 

• 

Add  a  question  that  focuses  on  the  "Paperless 
environment" . 

Item 

8.6: 

• 

Define  "MIS".  (2) 

Item 

8.7: 

• 

Define  external  support  or  delete.  (2) 

Item 

8.8: 

• 

Irrelevant  question,  delete. 

• 

Define  technical  support.  Is  this  the  equipment 

itself  or  the  tech  spt  group  within  the  PMO? 

• 

Look  at  asking  what  the  PMO  does  well  also. 

Item 

9.1: 

• 

Look  at  asking  what  the  PMO  does  well  also. 

Item 

9.2: 

• 

Look  at  asking  what  the  PMO  does  well  also. 
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Item  9.3: 


•  Look  at  asking  what  the  PMO  does  well  also. 


Additions : 

•  Item  in  group  1:  Does  the  Project  Office's  vision 
support  our  customers. 

•  Item  in  group  2:  What  causes  you  the  most  stress  in 
your  daily  job? 

•  Item  in  group  3:  Is  there  a  proper  balance  between 
functional  and  product  line  roles 
(responsibilities) ? 

•  Item  in  group  4:  What  makes  the  Integrated  Product 
Teams  (IPT)  work?  What  needs  to  be 

improved? 

•  Item  in  group  7:  Is  the  leadership/organization 
responsive  to  the  needs  of  the  customer? 

•  Item  in  group  8:  Do  you  feel  the  Project  office 

produces  quality  products,  provides  quality 
services? 

•  Item  in  group  8:  Do  you  feel  the  Project  office 

produces  quality  end  items  (HW/SW) ? 

•  Item  in  group  8:  How  effectively  do  you  feel  the 
Project  office  communicates,  works  with, 
contractors?  Other  customers? 

•  Item:  Focused  on  the  quality  of  the  counseling. 

•  Item:  Do  you  know  the  goals  of  the  PEO? 

•  Item:  How  satisfied  are  you  with  the  support  you 
receive  from  the  other  offices  within  the  PMO? 

•  Item:  To  what  extent  is  the  awards  system  limited 
by  the  command  (external  to  the  PMO) . 
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APPENDIX  H:  QUALITATIVE  RESPONSE  SUMMARY:  UNIT  SELF- 
ASSESSMENT  AND  SOFTWARE  SURVEY 


Unit  Self-assessment: 

Item  A4 :  What  questions/areas  do  you  feel  are  important 
indicators  of  program  health  that  are  not  included  in  the 
assessment? 

•  Relationships  with  matrix/external  organizations 
(possibly  under  structure) . 

•  Questions  that  focus  on  the  project  office's 
relationship  with  the  PEO  &  MICOM. 

•  As  noted  during  sample/walk  through  of  the 
assessment-worker  perceptions/knowledge  of  mission, 
goals,  vision,  and  their  role/position  in  achieving 
them.  How  is  their  worth  defined,  validated,  and 
preserved.  Do  they  make  a  difference?  Are  they 
part  of  the  team  ,  or  not? 

•  Training  -  computer  literate. 

•  Customer  satisfaction  related  questions  on  internal 
customers . 

•  Participation  in  "off-site"  activities.  The  turn 
out  we  have  shows  that  a  lot  of  employees  don’t 
want  to  participate. 

•  Leadership  and  technical. 

•  Add  assessment  of  individual  training. 

Item  A6:  Please,  provide  any  general  comments  you  may  have 

about  the  assessment  instrument. 

•  The  assessment  was,  with  the  exception  of  one 
question,  clear  and  concise.  Some  of  the  questions 
were  inappropriate  for  a  support  contractor  like 
myself,  but,  in  general,  the  assessment  will  be  a 
useful  tool  for  Project  Management  in  assessing  the 
level  of  success  of  their  leadership. 

•  Much  shorter  and  to  the  point  than  the  previous 
survey. 

•  The  assessment  was  quick,  easy  to  understand,  and 
appropriate  to  this  project  office. 

•  Very  positive.  I  think  the  tool  is  solid. 
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Asks  the  right  kind  of  questions  and  not  too  many. 
Is  not  tiring  and  aggravating  to  take. 

Easy  to  use,  painless. 

Minor  modifications  will  result  in  an  excellent 
program. 

I  think  the  assessment  is  a  good  way  to  say  what 
you  feel  or  think. 

Upgrade  notification  of  changes  in  processes/ 
procedures  to  ensure  everyone  needing  to  know  knows 
in  a  timely  manner,  rather  than  "piece  mill" 
notices . 

Make  the  questions  in  the  assessment  more 
personalized  (level  of  organization  (scope) ) . 

Good  -  it  gave  me  some  areas  that  I  feel  I  could 
really  improve  on. 

Will  be  a  good  indicator  of  program  office  health. 

Well  written,  thorough,  user  friendly,  applicable. 

It  has  good  functionality.  Easy  to  use  and 
understand. 

Some  of  the  questions  were  unclear,  relative  to  the 
answer  scale  provided.  I  feel  that  this  assessment 
could  have  some  value  if  used  properly. 

I  thought  it  was  a  good  assessment,  with  good 
questions . 

Well  planned  -  thought  out. 

The  assessment  needs  to  ask  questions  at  the 
division/section  level  and  not  always  refer  to  the 
project  office  as  a  whole. 

Is  on  the  mark.  Need  good  resume  function  to 
ensure  people  have  plenty  of  time  to  answer  word 
questions . 

The  assessment  addressed  all  disciplines. 

Delete  rewards  group.  The  rewards  policy  is  rigid 
and  cannot  be  changed  from  within. 

Delete  technology  group.  Project  is  automated. 
Not  worth  the  focus. 

Delete  rewards  and  technology  groups. 

Add  groups  that  focus  on  products  and  services. 
Change  Group  8  from  Technology  to  Products. 
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Task  group  (#2)  Scale:  moderately  doesn't  apply. 
All  of  the  questions  in  section  4  should  use  scale 
#2. 

Add  an  area  that  measures  the  effect  of  external 
influences  on  the  project  office  (PEO  and  MICOM) . 

Add  an  area  or  some  questions  about  how  well  the 
PMO  work  with  the  Contractor. 

Define  "free"  or  "open"  response  in  the 
instructions.  Suggest  "multiple  choice"  and  "short 
answer".  (2) 

Most  of  the  questions  are  to  general  (encompassing 
the  entire  PMO) .  Most  people  will  not  have  a  clue 
at  this  level.  Focus  all  questions  down  to  the 
personal6  or  office  level.  The  analysis  of  all 
responses  will  give  an  accurate  picture  of  the 
entire  office. 

Barely  selection  on  scale  #1  might  be  better  as 
Rarely. 

Consider  allowing  decimal  responses  (in  between  two 
answers) . 

Emphasize  PMO  internal  answers. 

Don't  care  for  open  ended  questions.  Stress 
importance  of  these  questions  in  instructions. 

I  have  yet  to  see  the  use  of  the  words  Value  Added 
or  Good  Enough.  Look  at  incorporating  these 
phrases . 

Informal  structure  incorporates  Integrated  Product 
Teams  (IPT) . 

Personalize  all  questions. 

Open  questions  need  to  solicit  positive  comments  as 
well.  (2) 

Focus  questions  to  the  division,  branch  level. 

Stress  confidentiality  of  information.  Possibly 
incorporate  a  letter  from  the  PM  backing  the 
project  and  stressing  confidentiality. 

Asks  nothing  about  our  reason  for  being!  Project 
offices  supply  Products  (HW,  SW,  Tng  &  Maintenance 
equipment,  and  documentation  to  the  soldier)  and 
Services  (technical  support,  field  services,  new 
material  studies  and  analysis,  service  life 
surveillance,  depot  support,  etc.,  to  soldiers  in 
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the  field,  as  well  as  to  HQDA  and  TRADOC)  .  The 
purpose  of  this  survey  is  to  see  how  well  we  are 
doing  this  job  and  whether  we  are  on  a  path  of 
continuous  improvement. 

•  The  words  Reasonably  and  Moderately  are  difficult 
to  differentiate  between.  Consider  using  alternate 
descriptions . 

•  Mix  and  match  selective  groups  of  questions. 


Software: 

Item  B5  (part  2) :  Did  you  encounter  any  difficulties  using  the 
software?  If  yes,  please  describe  the  difficulties. 

•  No  response 

Item  B6:  What,  if  any,  improvements  to  the  software  would  you 
suggest? 

•  Streamline  the  installation  process  to  eliminate 
the  additional  "install"  and  icon  setup  portions. 
This,  however,  is  not  a  major  problem.  Also,  you 
might  look  for  a  way  to  disable  the  comma  key 
during  text  entry  so  as  to  eliminate  the  problems 
with  comma  delimited  files. 

•  Add  default  buttons  for  the  next  screen.  Add 
infinite  scale  for  answers. 

•  Allow  the  user  to  back  up  to  previous  question. 

•  The  wait  between  question  groups  needs  to  be 
eliminated. 

Item  B7 :  Please,  provide  any  general  comments  you  may  have 
about  the  software. 

•  Software  worked  well  -  the  windows  were 

aesthetically  pleasing.  The  arrow  method  for 
selecting  responses  was  different,  but  easy  and 
pleasant  to  use. 

•  The  software  was  clear  and  concise.  Made  the 
assessment  easier  to  answer  in  short  period  of  time. 

•  I  think  that  the  software  is  great  and  I  really 
enjoyed  using  it. 

•  Some  of  the  bar  scales  should  be  the  strongly  agree 
scale . 
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Instead  of  dragging  the  arrow  to  answer  the 
questions,  let  me  click  on  the  answer  and  go. 

Easy  to  use.  Good  choice  of  color  scheme  -  easy  to 
read. 

Has  a  professional  "look".  Makes  respondents  have 
more  confidence  in  results. 

User  friendly.  Well  written.  (4) 

Using  the  windows  software  is  certainly  a  nice 
innovation.  However,  the  facilitator  is  obligated 
to  provide  the  general  instructions  repetitively, 
therefore,  more  time  consuming  when  compared  to  all 
respondents  at  a  single  session.  (Latter  half  of 
comment  retracted  when  informed  about  networking 
the  software  and  automating  the  instructions.) 

Good  package. 

Add  Senior  Programmer  to  the  position  listing. 

Include  "N/A"  or  "Other"  in  all  lists. 

Unclear  on  "time  in  position"  field.  Does  it  refer 
to  only  to  duty  in  current  office  or  type  of  duty 
position? 

Need  to  be  able  to  browse  through  questions  both 
forward  and  backwards.  (17) 

Must  be  careful  when  dragging  arrow.  If  you 
release  late,  it  could  change  your  answer. 
Mention  in  instructions. 

Need  to  specify  "Years"  for  time  demographic 
fields.  (5) 

Might  be  better  to  ask  for  DOB  vice  age. 

Scale  might  be  easier  to  use  (dragging  arrow)  if  it 
was  sideways  on  the  screen. 

Management  vs  others  is  an  important  comparison  for 
analysis . 

Difficult  to  switch  between  instructions  and 
assessment . 

Need  to  find  a  way  to  have  the  program  disregard 
the  commas  in  the  text  fields  so  the  user  can 
type  normally,  else  find  a  different  way  to  export 
or  analyze  the  data.  (12) 

The  year  field  must  be  defined.  Is  it  the  year  you 
are  taking  the  assessment?  (2) 


Need  to  let  the  user  know  that  it  is  the  office 
symbol  you  are  looking  for  in  the  office  field. 

Combine  GS  and  GM  to  GS/GM  in  the  rank  field. 

Is  it  possible  to  let  the  user  select  an  answer 
that  is  in  between  those  offered  on  the  scale? 

Adjust  the  main  screen  to  include  AGMS  and  the  PEO. 

Add  "Logistics  Management  Specialist"  to  the 
position  list.  (2) 

Is  the  time  in  the  organization  field  based  on 
cumulative  time  in  the  organization  (what  about 
people  that  have  been  assigned  there  twice  or 
more?)  (2) 

Consider  tailoring  certain  groups  of  questions  to 
be  viewed  only  by  certain  groups  of  people.  For 
example  Management  Questions  (do  not  appear  to  all 
others) . 

Add  the  position  Support  Clerk  to  the  position 
list . 

Does  time  in  organization  refer  to  Government  or 
the  PMO?  (2) 

Add  contract  specialist  to  the  position  list. 

Add  below  high  school  in  the  education  field. 

Add  Branch  Chief  to  the  position  list. 

Add  years  of  college,  other  than  degree  to  the 
education  list.  (3) 

Configure  the  program  to  enable  clicking  on  the 
answers  instead  of  dragging  the  arrow  across  the 

screen.  (14) 

Add  program  analyst  to  the  position  list. 

Add  SFAE-MSL-HD-P  to  the  office  listing  (scrub  the 
office  list  against  the  new  wire  diagram) . 

Incorporate  an  alert  message  to  remind  users  of 
their  respondent  number  upon  exiting  the  program 
from  any  place,  except  the  end  of  the 
questionnaire . 

Add  Electronics  Engineer  to  the  position  list.  (2) 

Add  Systems  Administrator  to  the  position  list. 

Consider  two  configurations:  A  Stand  alone  and 
a  network  version. 
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Add  General  Engineer  to  the  position  list. 

Add  Configuration  Management  Specialist  to  the 
position  listing.  (2) 

Consider  adding  ages  greater  than  65  to  the  age 
listing. 

Add  Operations  Research  Analyst  to  the  position 
listing. 

Are  you  asking  for  the  percent  of  the  office  or  the 
percent  of  the  individual  definitization? 

Add  a  Core/Matrix  demographic  field.  (5) 


APPENDIX  I:  PROJECT  MANAGER  INTERVIEW 


Name:  _  Organization: 


l.  What  do  you  intend  to  do  with  the  results  of  the 
assessment? 


2 .  What  types  of  analyses  of  the  assessment  data  would 
be  most  beneficial  to  you? 


3.  What  types  of  outputs  (presentations,  reports, 
etc.)  would  be  most  beneficial  to  you? 


4.  How  do  you  define  program  health? 


5.  In  your  opinion,  is  the  assessment,  as  written, 
focused  on  the  indicators  that  determine  the  program  health 
of  your  organization? 


6.  What  areas  of  program  health  are  not  covered  in  the 
assessment? 
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7.  At  what  level (s)  do  you  intend  to  conduct  this 
assessment  at  in  the  future? 


8.  Do  you  intend  to  benchmark  your  organization 
against  other  organizations  in  the  future  using  this 
assessment?  If  yes,  which  organizations? 


9.  Does  the  Self-assessment  package  (assessment  and 
software)  provide  you  with  the  tools  necessary  to  identify 
weaknesses  in  your  organization  and  monitor  their 
improvement? 
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Establish  baseline  for  comparison  of  future  repetitive 
test  responses  for  the  purpose  of  recognizing  trends  and 
focusing  on  quality  improvement. 


•  Evaluate  the  PMO.  Establish  trends  in  the  PMO. 

•  Analyze  the  results  to  determine  the  health  of  the 

organization.  Pinpoint  areas  to  work  on,  such  as 
communications,  procedures,  etc... 

Item  2.  What  types  of  analyses  of  the  assessment  data  would  be 
most  beneficial  to  you? 

•  Each  category  (mission,  task,  etc...)  with  actual 

results,  ability  to  compare  to  historical  data,  trend 
analysis.  Discuss  accuracy  and  model  sensitivities. 

•  Comparisons  of  demographically  significant  subsets  of  the 
PMO  against  the  total  population  and  against  each  other 
(i.e.  each  division/branch  office  against  the  overall 
scores,  management  against  all  others) .  Benchmark 
against  previous  years  (five  years  total) . 

•  Big  three  or  four  problem  areas  -  what  are  they?  And  at 
what  levels  in  the  organization  -  senior  leadership,  mid¬ 
level  managers,  worker  bees?  Is  age  versus  grade/level 
the  biggest  differentiation  in  the  answers  we  get?  Are 
"don't  knows"  a  significant  percentage  of  the  responses? 

Item  3.  What  types  of  outputs  (presentations,  reports,  etc...) 
would  be  most  beneficial  to  you? 

•  Statistics  and  bar  charts.  Carry  up  to  five  data  samples 
(Current  plus  past  four) . 

•  Statistics  and  bar  charts  summarizing  and  comparing  the 
data .  Current  year  against  previous  years .  Management 
against  all  others.  Offices  against  offices  and  the 
total  population.  Percentage  scored  against  possible. 
Not  concerned  with  age,  education  level,  or  military 
versus  civilian. 

•  Graphs  of  answers  by  rank,  age,  organization,  and 
product.  Trend  charts  -  most  people  that  thought  "a"  was 
screwed  up  also  thought  "b"  was  screwed  up. 
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Item  4 . 


How  do  you  define  program  health? 


Item  5 


Item  6 


Item  7 


Item  8 


Success  in  fielding  and  supporting  material  to  the 
soldier. 

Job  getting  done,  people  like  the  job,  they  have  the 
tools  they  need,  the  future  is  reasonably  clear,  rewards 
are  substantial  for  the  deserving,  people  think  the  front 
office  cares,  people  think  they  have  authority  without 
"mother  may  I",  dead  wood  gone. 

In  your  opinion,  is  the  assessment,  as  written,  focused 
on  the  indicators  that  determine  the  program  health  of 
your  organization? 

Not  yet.  Need  to  measure  success  of  our  ultimate 
products  (material  and  services) . 

For  the  most  part.  We  have  outside  influences  (PEO 
structure,  MICOM,  contractors)  that  aren't  really 
addressed. 

What  areas  of  program  health  are  not  covered  in  the 
assessment? 

Products  and  Services . 

Customer  satisfaction,  Integrated  product  Teams  (IPT) . 

Do  the  people  think  the  program  they  are  working  on  (i.e. 
ITAS,  IBAS,  etc...)  are  on  track? 

At  what  level (s)  do  you  intend  to  conduct  this 
assessment  at  in  the  future? 

All  levels  in  the  PMO  (100%) .  (2) 

100%  (verbal  response  extracted  from  initial  guidance) . 

Do  you  intend  to  benchmark  your  organization  against 
other  organizations  in  the  future  using  this  assessment? 
If  yes,  which  organizations? 

All  projects  in  the  PEO,  ultimately.  Already  done  once, 
using  inappropriate  tool . 

Not  concerned  with  benchmarking  against  other  projects. 
Focus  is  on  benchmarking  against  previous  years  (verbal 
response  extracted  from  initial  guidance) . 
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Item  9 


I 


Not  necessarily  -  only  several  other  organizations  use 
exactly  the  same  survey. 

Does  the  Self-assessment  package  (assessment  and 
software)  provide  you  with  the  tools  necessary  to 
identify  weaknesses  in  your  organization  and  monitor 
their  improvement? 

Yes. 

Depends  on  what  the  analyses  look  like.  Too  soon  to 
tell . 
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APPENDIX  K:  UNIT  FACILITATOR  INTERVIEW 


Name:  _  Organization:  _ 

1.  Compared  to  the  last  assessment  you  facilitated,  to 
what  extent  did  the  software  change  your  role  as 
facilitator?  In  what  way(s)? 


2 .  Do  you  think  you  will  be  able  to  manage  the  data 
collection  and  analysis  process  once  the  software  is 
installed  on  the  organization's  network  and  each  respondent 
completes  the  assessment  from  their  individual  work 
stations? 


3.  What  difficulties,  if  any,  do  you  expect  to 
encounter? 


4.  Do  you  think  the  software  procedures  for  gathering, 
compiling,  exporting,  and  analyzing  data  are  easier  or 
harder  than  doing  these  tasks  manually?  In  what  way? 


5 .  Are  the  instructions  for  manipulating  data  using 
the  software  complete?  Understandable? 
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6.  Were  you  able  to  import  the  assessment  output  file 
into  your  current  analysis  program (s)  to  conduct  an  analysis 
on  the  assessment  data?  What  difficulties,  if  any,  did  you 
encounter? 


7.  Which  analysis  program(s)  do  you  intend  to  use  with 
this  data  in  the  future? 


8 .  What  types  of  analyses  do  you  intend  to  conduct  on 
this  data  in  the  future? 


9.  What  types  of  outputs  (presentations,  reports, 
etc..)  do  you  intend  to  prepare  using  this  data  in  the 
future? 


10 .  Are  the  instructions  for  manipulating  the 
assessment  data  using  the  software  complete? 
Understandable? 


APPENDIX  L:  QUALITATIVE  RESPONSE  SUMMARY:  UNIT  FACILITATOR 
INTERVIEWS 

Item  1.  Compared  to  the  last  assessment  you  facilitated,  to  what 
extent  do  you  expect  the  software  change  your  role  as 
facilitator?  In  what  way(s)? 

•  Facilitator  much  less  involved  in  filling  out  the  survey. 
Keeps  facilitator  bias  out.  Much  better.  More  tailored 
to  the  PMO .  Easier  on  facilitator. 

•  It  will  make  the  analysis  of  the  data  easier,  especially 
if  it  is  fully  automated.  It  will,  however,  be  difficult 
to  manage  who  has  or  has  not  responded  to  the  survey  at 
any  given  time. 

Item  2.  Do  you  think  you  will  be  able  to  manage  the  data 
collection  and  analysis  process  once  the  software 
is  installed  on  the  organization's  network  and  each 
respondent  completes  the  assessment  from  their  individual 
work  stations? 

•  Should  not  be  too  difficult. 

•  No,  it  will  be  extremely  difficult  to  manage  who  has  or 
has  not  completed  the  survey  based  solely  on  the  number 
of  records  in  the  database. 

Item  3.  What  difficulties,  if  any,  do  you  expect  to  encounter? 

•  Tailoring  analyses  to  detect  demographic  differences. 

Item  4.  Do  you  think  the  software  procedures  for  gathering, 

compiling,  exporting,  and  analyzing  data  are  easier  or 
harder  than  doing  these  tasks  manually?  In  what  way? 

•  Should  be  easier  than  manual.  Have  not  used  or  viewed 
the  analysis  sub-program. 

•  It  will  be  much  easier  logistically.  Also,  the  compiling 
and  analysis  of  the  data  should  be  much  easier  on  the 
facilitator. 

Item  5.  Are  the  instructions  for  manipulating  data  using  the 
software  complete?  Understandable? 


Item 


Item 


Item 


Item 


Item 


•  The  instructions  were  not  complete  at  the  time  of 
testing. 

6.  Are  you  familiar  with  the  steps  involved  in  importing  a 
comma  or  tab  delimited  text  output  file  into  your  current 
analysis  program (s)  to  conduct  an  analysis  on  the 
assessment  data?  What  difficulties,  if  any,  did  you 
encounter? 

•  No.  (2) 

7.  Which  analysis  program (s)  do  you  intend  to  use  with  this 
data  in  the  future? 

•  Microsoft  Office 

•  Would  wish  the  analysis  be  a  plotting  sub-routine.  Don't 
want  just  raw  data. 

•  MS  Office  is  installed  on  our  computers.  Program  written 
by  the  researcher  would  be  fine. 

8.  What  types  of  analyses  do  you  intend  to  conduct  on  this 
data  in  the  future? 

•  Overall  (general  statistics) .  Historical  comparison  by 

item  number  to  determine  trends.  Comparisons  by 
demographic  groups,  if  necessary,  to  identify,  focus  in, 
on  problem  areas . 

•  That  is  a  decision  for  top  management. 

9.  What  types  of  outputs  (presentations,  reports,  etc..)  do 
you  intend  to  prepare  using  this  data  in  the  future? 

•  Would  use  plots  to  brief  top  management. 

•  Statistical  tables  and  bar  charts.  Anything  that  will 
effectively  demonstrate  the  areas  that  top  management  has 
said  they  want  to  see. 

10.  Are  the  instructions  for  manipulating  the  assessment  data 
using  the  software  complete?  Understandable? 

•  The  instructions  were  not  complete  at  the  time  of 
testing. 
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Item  11.  How  much  control  do  you  desire  to  have  over  the 
facilitator  (DBASE,  analysis  functions)? 

•  Would  like  a  fully  automated,  canned,  analysis  program. 

•  Would  like  both  a  canned,  fully  automatic,  analysis 
program  and  the  ability  to  generate  additional  queries 
from  the  database  and  plot  them. 
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APPENDIX  H:  INSTRUCTIONS:  THE  CCAWS/AGMS  UNIT  SELF-ASSESSMENT 


1.  The  CCAWS/AGMS  Unit  Self-assessment  is  composed  of  77  items 
divided  into  eleven  groups .  There  are  two  types  of  items  used  in 
the  assessment:  multiple  choice  and  short  answer.  Multiple  choice 
items  are  answered  by  placing  the  sliding  arrow  opposite  the 
desired  response  from  the  selections  provided.  Each  multiple 
choice  item  is  rated  on  one  of  the  following  scales . 


Scale  1 

Value 

Scale  2 

Totally 

<  6  ► 

Strongly  Agree 

Substantially 

+  5  ► 

Agree 

Reasonably 

<  4  ► 

Slightly  Agree 

Moderately 

«  3  ► 

Slightly  Disagree 

Barely 

«  2  ► 

Disagree 

None 

«  1  ► 

Strongly  Disagree 

Don ' t  Know 

<  0  ► 

Don ' t  Know 

Note:  "Don't  Know"  is  not  a  neutral  response. 

2 .  The  second  type  of  items  used  are  short  answer  items .  The 
value  of  the  assessment  is  largely  dependent  upon  the  quality  of 
the  responses  to  these  items.  These  items  are  answered  by  typing 
your  response  into  the  yellow  box  provided. 

3 .  Before  you  begin  the  assessment  you  will  be  asked  to  provide 
demographic  information  about  yourself.  This  information  is 
solicited  solely  to  provide  information  for  use  in  the  analysis  of 
the  assessment  items.  This  information  is  requested  on  a  volunteer 
basis . 

4.  Instructions  on  the  use  of  the  assessment  software  are  included 
as  a  separate  document . 
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APPENDIX  N:  INSTRUCTIONS:  THE  CCAWS/AGMS  UNIT  SELF-ASSESSMENT 
SOFTWARE 

1.  General.  The  CCAWS/AGMS  Unit  Self-assessment  software  is  a 
prototype  application  independently  developed  for  the  purpose  of 
conducting  the  CCAWS/AGMS  Unit  Self-assessment. 

2.  System  Requirements. 

a.  IBM  or  compatible  computer  system. 

b.  386SX-25  or  faster  CPU. 

C .  4MB  RAM . 

d.  7.5MB  hard  drive  (free  space). 

e .  MS  DOS  5.0  or  higher . 

f .  MS  Windows  3 . 1  or  higher . 

g.  MS  Mouse  or  equivalent  pointing  device. 

h.  640x480  resolution  video  driver  or  higher. 

3.  Software  Installation.  The  following  procedures  are  used  to 
install  the  unit  assessment  software  onto  your  computer  system: 

a.  Ensure  Microsoft  Windows  is  running  on  your  computer. 

Note:  This  program  cannot  run  outside  of  the  MS  Windows 

environment . 

b.  Insert  the  disk  labeled  Disk  1  into  your  3.5"  floppy  disk 
drive . 

c.  From  the  File  menu  select  Run. 

d.  Type  the  drive  letter  of  your  3.5"  floppy  disk  drive 
followed  by  Setup.exe,  (example  -  A:  Setup.exe)  ,  then  press  enter. 

e.  You  will  be  asked  if  you  are  sure  you  want  to  install  the 
unit  assessment  program  onto  your  computer  system.  Select  "Yes"  if 
you  wish  to  proceed  with  the  installation.  To  terminate  the 
installation,  select  No  or  Cancel. 

f.  You  will  be  prompted  to  enter  the  drive  letter  or  network 
volume  ID  to  which  you  wish  to  install  the  unit  assessment  program 
files.  Enter  this  information  in  the  space  provided. 
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g.  When  prompted,  insert  the  requested  discs  into  the  3.5" 
floppy  disk  drive,  then  press  OK. 

The  unit  self-assessment  setup  program  creates  a  directory 
entitled  assess  under  the  root  directory  of  the  drive  or  network 
volume  you  indicated  and  then  transfers  the  unit  assessment  program 
files  to  this  directory.  The  setup  program  will  then  create  a  Unit 
Self-assessment  program  group  and  a  CCAWS/AGMS  Unit  Self-assessment 
program  item  (icon)  within  the  Windows  Program  Manager.  Note:  if 
installing  this  program  on  a  Local  Area  Network  (LAN)  ,  this 
procedure  must  be  repeated  manually  at  each  workstation  from  which 
you  wish  to  access  to  the  assessment  program.  The  procedure  to 
accomplish  this  is: 

1.  Ensure  MS  Windows  is  running  on  the  desired  workstation. 

2.  From  the  File  menu,  choose  New. 

3.  Select  Program  Group. 

4.  Type  Unit  Self-Assessment  on  the  title  line. 

5 .  Press  OK. 

6 .  Repeat  Step  2 . 

7.  Select  Program  Item. 

8.  Press  the  Browse  button. 

9 .  Select  the  directory  or  volume  ID  in  which  you  stored  the 
assessment  program  files. 

10.  Select  the  assess  directory. 

11.  Select  the  file  entitled  assess.exe. 

12 .  Press  OK. 

4.  Starting  the  Assessment  Program.  The  unit  assessment  program 
is  started  by  "double-clicking"  the  CCAWS/AGMS  Unit  Self-Assessment 
program  icon  with  the  left  mouse  button. 

5.  Beginning  a  New  Assessment. 

a.  Start  the  Unit  Assessment  Program  (Step  4) . 

b.  Press  the  Begin  Assessment  button  in  the  main  program 
window . 
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c.  Enter  the  appropriate  demographic  information  in  the 
spaces  provided.  Note:  the  demographic  entries  are  fixed  response 
items.  To  choose  your  response,  click  with  your  left  mouse  button 
on  the  arrow  to  the  right  of  the  space  provided  to  reveal  a  drop¬ 
down  listing  of  available  responses.  Scroll  through  the  available 
responses  by  clicking  on  the  up  or  down  arrows  to  the  right  of  the 
listing  until  you  find  the  appropriate  response.  Select  the 
response  by  clicking  on  it  with  the  left  mouse  button. 

d.  Press  the  Continue  button. 

7.  Conducting  the  Assessment.  As  stated  in  the  CCAWS/AGMS  Unit 
Self-assessment  instructions  (separate  document) ,  the  assessment 
incorporates  two  types  of  assessment  items:  multiple  choice  and 
shot  answer.  The  following  procedures  are  used  to  respond  to  the 
types  of  items  used  in  the  assessment. 

a.  Multiple  Choice.  Multiple  choice  items  asks  a  question, 
or  provides  a  statement,  to  which  there  are  a  limited  number  of 
fixed  responses.  These  items  are  accompanied  by  one  of  two  scales 
(see  CCAWS/AGMS  Unit  Self-assessment  instructions)  with  a  blue 
arrow  to  the  right  of  the  scale,  positioned  across  from  the  Don't 
Know  response.  There  are  three  methods  of  selecting  the 
appropriate  response  to  this  type  of  question. 

(1)  Select  the  appropriate  response  using  the  mouse  or 
pointing  device. 

(2)  Drag  the  arrow  up  or  down  the  scale  using  the  mouse 
or  pointing  device  until  it  is  across  from  the  desired  response. 

(3)  Position  the  arrow  across  from  the  desired  response 
by  moving  it  up  or  down  the  scale  using  the  arrow  buttons  on  your 
keyboard . 

Note:  It  is  not  necessary  to  move  the  arrow  if  the  Don't  Know 
response  is  appropriate. 

b.  Short  Answer.  Short  answer  items  allow  the  respondent  to 
input  a  "free",  unrestricted  response  to  the  stated  item.  To 
answer  this  type  of  item,  simply  type  the  information  into  the 
yellow  box  provided. 
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c.  Once  you  have  answered  each  question  press  the  Continue 
button  located  in  the  lower  right-hand  corner  of  each  item  window 
to  go  to  the  next  item. 

7 .  Exiting  the  program.  The  program  can  be  exited  in  several 
ways . 

a.  Press  the  Exit  button  in  the  main  program  window.  Note: 
this  window  only  appears  at  the  beginning  and  the  end  of  the 
assessment  process. 

b.  "Double-click"  the  minus  sign  located  in  the  upper  left- 
hand  corner  of  any  item  window. 

c.  Choose  Quit  from  the  Facilitator  menu  (Facilitator  only) . 

8.  Resuming  a  Previous  Assessment.  The  following  procedures  are 
used  to  resume  a  previous  assessment. 

a.  Start  the  unit  assessment  program  (Step  4) . 

b.  Press  the  Resume  Assessment  button  in  the  main  program 
window . 

c.  Enter  your  Respondent  Number  in  the  upper  box.  IMPORTANT! 
You  must  use  the  same  respondent  number  you  received  when  taking 
the  initial  assessment.  This  number  is  provided  to  you  when  you 
exit  the  original  assessment.  If  you  cannot  remember  this  number, 
return  to  the  main  program  window  and  begin  a  new  assessment. 

d.  Press  Tab. 

e.  Enter  the  number  that  corresponds  to  the  item  group  that 
you  would  like  to  begin  (i.e.  item  group  4=4). 

f.  Enter  the  number  that  corresponds  to  the  item  that  you 
would  like  to  begin  (i.e.  item  3=3). 

f.  Press  the  Resume  button. 

g.  Resume  answering  the  remaining  questions  using  the 
procedures  in  step  6 . 
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9 .  Facilitator  Functions . 

1.  Exporting  the  Data  to  MS  Access  for  Analysis: 

a.  Start  the  unit  assessment  Program  (Step  4) . 

b.  Press  the  Facilitator  Functions  button  in  the  main 
program  window. 

c.  Enter  the  security  password,  then  press  enter. 

d.  Choose  Export  in  the  Facilitator  menu. 

Note:  the  unit  self-assessment  program  exports  the  demographic  and 
item  data  into  a  tab  delimited  text  file.  The  file  is  entitled 
trnsfr.  txt  and  is  placed  in  the  directory  containing  the  unit 
assessment  program  files.  The  unit  self-assessment  program  is 
designed  to  automatically  launch  MS  Access,  then  import  and  format 
the  data  using  this  program.  Users  of  database  and  analysis 
programs  other  than  MS  Access  can  import  data  from  this  file 
manually,  according  to  the  program's  procedures. 

2.  Data  Analysis.  Calculations  required  to  analyze  the 
collected  data  are  automatically  performed  as  the  data  are  imported 
into  MS  Access.  To  perform  additional  analyses  on  the  data,  refer 
to  the  MS  Access  users  guide. 

3.  Presenting  the  Analyzed  Data.  The  presentation  function 
within  the  data  storage,  analysis,  and  presentation  application  is 
designed  to  automatically  present  the  graph  depicting  the  overall 
assessment  results,  by  organization  and  year.  The  application,  as 
delivered,  is  capable  of  presenting  five  separate  levels  of 
analyzed  data.  Each  level  contains  greater  detail  than  the 
previous  level.  To  view  each  successive  level  of  the  analysis, 
simply  press  the  push  button  provided  that  corresponds  to  the  area 
in  which  you  would  like  to  focus.  For  example:  The  initial 
presentation  screen  provides  an  overall  assessment,  depicting  the 
results  of  each  unit  within  the  database  for  each  year  the 
assessment  has  been  taken.  If  you  would  like  to  view  the  CCAWS 
organization  in  more  detail,  you  would  depress  the  CCAWS  button. 
If,  however,  you  would  like  to  compare  each  organization  based  on 
the  1995  results,  you  would  press  the  1995  button.  This  procedure 
is  repeated  at  each  succeeding  level  of  analysis.  Analyzed  data 
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can  be  viewed  with  respect  to  each  organization,  office, 
demographic  category,  and  year,  or  any  combination  of  these  areas. 
Copies  of  each  presentation  screen  can  be  made  by  pressing  the 
print  button,  located  on  each  screen. 
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APPENDIX  O:  DATA  SOURCE  MATRIX 
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granted  June  13,  1995. 

Heatherington,  Billy,  Assistant  Project  Manager  (Product 
Assurance,  Test  and  Evaluation) ,  Close  Combat  Anti-armor 
Weapon  Systems,  PEO  Tactical  Missiles,  Interview  granted 
September  21,  1994. 
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Higginbotham,  Claude,  Chief,  Horizontal  Technology 
Integration  Branch,  Close  Combat  Anti-armor  Weapon  Systems, 
PEO  Tactical  Missiles,  Interview  granted  June  14,  1995. 

Hunt,  V.  Daniel,  Quality  Management  for  Government:  A  Guide 
to  Federal.  State,  and  Local  Implementation.  Milwaukee,  WI : 
ASQC  Quality  Press,  1993. 

Ishikawa,  Kaoru,  What  is  Total  Quality  Control?  the  Japanese 
Way.  New  York:  Prentice-Hall,  1985. 

Jacovides,  George,  Electronics  Engineer,  Air-to-Ground 
Missile  Systems,  PEO  Tactical  Missiles,  Interview  granted 
June  21,  1995. 

Joyner,  Tom,  Logistics  Specialist,  Air-to-Ground  Missile 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  21, 
1995. 

Juran,  J.  M.,  Juran  on  Leadership  for  Quality:  An  Executive 
Handbook.  The  Free  Press,  1989. 

Kerzner,  Harold,  Project  Management:  A  System  Approach  to 
Planning.  Scheduling  and  Controlling.  4th  ed.  New  York:  Van 
Nostrand  Reinhold.  1992. 

Lawson,  Gary,  Assistant  Project  Manager  (Production) ,  Close 
Combat  Anti-armor  Weapon  Systems,  PEO  Tactical  Missiles, 
Interview  granted  September  22,  1994. 

Leonard,  Frank  S.,  and  Sasser,  Earl  W.  "The  Incline  of 
Quality,"  Harvard  Business  Review.  September-October  1982. 

Liston,  Ruth,  Secretary,  Air-to-Ground  Missile  Systems,  PEO 
Tactical  Missiles,  Interview  granted  June  20,  1995. 

Lowe,  Jon,  Deputy  Project  Manager,  Close  Combat  Anti-armor 
Weapon  Systems,  PEO  Tactical  Missiles,  Interview  granted 
September  19,1994;  June  13,  1995. 

Mansi,  Brian  E.,  and  Schacht,  Nicholas  R.  Total  Quality 
Management.  A  Guide  to  Implementation.  Washington,  D.C.: 
Asst.  Secretary  of  Defense  for  Production  and  Logistics, 
1989,  DLSIE  LD084024A. 

Maples,  Henri,  Logistics  Specialist,  Air-to-Ground  Missile 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  19, 
1995. 
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McGovern,  John  P.,  "The  Evolution  of  Total  Quality 
Management,"  Program  Manager.  September-October  1990. 

Meadows,  Petey,  Secretary,  Close  Combat  Anti-armor  Weapon 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  15, 
1995. 

Microsoft  Corporation,  User's  Guide,  Microsoft  Access,  1994. 

Microsoft  Corporation,  Getting  Started/Building 
Applications.  Microsoft  Access.  1994. 

Middlebrooks,  Leland,  Chief,  Program  Support  Division,  Close 
Combat  Anti-armor  Weapon  Systems,  PEO  Tactical  Missiles, 
Interview  granted  September  20,  1994;  June  13,  1995. 

Mitchell,  Chuck,  Vice  President,  Systems  Improvement  Inc., 
Colorado  Springs,  CO,  Interview  granted  March  22,  1995. 

Mizuno,  Shigeru,  Management  for  Quality  Improvement. 
Cambridge,  MA:  Productivity  Press,  1988. 

Morris,  Richard,  Lieutenant  Colonel,  U.S.  Army,  Product 
Manager,  Hellfire  II,  Air-to-Ground  Missile  Systems,  PEO 
Tactical  Missiles,  Interview  granted  June  22,  1995. 

National  Institute  of  Standards  and  Technology,  Malcolm 
Baldridae  Quality  Award  Application.  Gaithersburg,  Maryland, 
1991. 

Nichols,  William,  Lieutenant  Colonel,  U.S.  Army,  Product 
Manager,  Improved  Bradley  Acquisition  System,  Close  Combat 
Anti-armor  Weapon  Systems,  PEO  Tactical  Missiles,  Interview 
granted  September  19,  1994;  June  14,  1995. 

Novell,  Inc.,  AppWare  Application  Development  Tools, 
Software,  November  1994. 

Novell,  Inc.,  AppWare  Users  Guide  (Second  Edition),  November 
1994. 

Parham,  Ollie,  Secretary,  Air-to-Ground  Missile  Systems,  PEO 
Tactical  Missiles,  Interview  granted  June  21,  1995. 

Parker,  Dan,  Senior  Contracting  Officer  for  the  Close  Combat 
Anti-armor  Weapon  Systems,  PEO  Tactical  Missiles,  Interview 
granted  November  28,  1994. 
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Polly,  Judy,  Secretary,  Close  Combat  Anti-armor  Weapon 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  14, 
1995. 

Porter,  Freddie,  Chief,  Configuration  Management  Branch, 
Air-to-Ground  Missile  Systems,  PEO  Tactical  Missiles, 
Interview  granted  June  22,  1995. 

Rausch,  Dave,  Operations  Research  Analyst,  Air-to-Ground 
Missile  Systems,  PEO  Tactical  Missiles,  Interview  granted 
June  22,  1995. 

Ray,  Richard,  Logistics  Specialist,  Close  Combat  Anti-armor 
Weapon  Systems,  PEO  Tactical  Missiles,  Interview  granted 
June  15,  1995. 

Readus,  Nancy,  Configuration  Management  Specialist,  Air-to- 
Ground  Missile  Systems,  PEO  Tactical  Missiles,  Interview 
granted  June  22,  1995. 

Redrick,  Beverly,  Secretary,  Close  Combat  Anti-armor  Weapon 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  15, 
1995. 

Riggs,  Kathy,  Secretary,  Air-to-Ground  Missile  Systems,  PEO 
Tactical  Missiles,  Interview  granted  June  22,  1995. 

Robbins,  Margaret,  and  Cristmann,  Roy,  APPWARE :  A 
Developer's  Guide,  New  York,  NY:  M&T  Books  1995. 

Saraph,  Jayant  V.,  Benson,  P.  George,  and  Schroeder,  Roger 
G.,  "An  Instrument  for  Measuring  the  Critical  Factors  of 
Quality  Management, ”  Decision  Sciences.  Vol.  20,  1989. 

Sanderson,  Tommy,  Senior  Programer,  Close  Combat  Anti-armor 
Weapon  Systems,  PEO  Tactical  Missiles,  Interview  granted 
September  22,  1994;  June  12,  1995. 

Schild,  Marilyn,  Chief,  Program  Management  Branch,  Air-to- 
Ground  Missile  Systems,  PEO  Tactical  Missiles,  Interview 
granted  June  16,  1995. 

Schoolfield,  Robert,  Development  Engineer,  Systems 
Improvement  Inc.,  Colorado  Springs,  CO,  Interview  granted 
March  24,  1995. 

Simpson,  Diane,  Electronics  Engineer,  Air-to-Ground  Missile 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  20, 
1995. 
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Systems  Improvement  Inc.,  CI-Toolkit  for  Windows,  Software, 
1995. 

Systems  Improvement  Inc.,  CI-Trainer,  Software,  1995. 

Systems  Improvement  Inc.,  Demonstration  and  User  Guide,  CI- 
Toolkit  for  Windows,  1995. 

Systems  Improvement  Inc.,  CI-Toolkit  Four  Day  Workshop 
Students  Notebook,  CI-Toolkit  for  Windows,  1995. 

Systems  Improvement  Inc.,  CI-Toolkit  Workshop  Instructors 
Notebook,  CI-Toolkit  for  Windows,  1995. 

Treshansky,  Joel,  Chief  Master  Sergeant  (Retired),  U.S.  Air 
Force,  Project  Manager,  SIMCO  Electronics,  Interview  granted 
June  9,  1995. 

Turner,  Lyell,  System  Administrator,  Air-to-Ground  Missile 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  20, 
1995. 

Utterback,  Jim,  Engineer,  Air-to-Ground  Missile  Systems,  PEO 
Tactical  Missiles,  Interview  granted  June  20,  1995. 

Walpole,  Ronald  E.,  and  Myers,  Raymond  H.,  Probability  and 
Statistics  for  Engineers  and  Scientists  (Fifth  Edition) ,  New 
York,  NY:  Macmillan  Publishing  Co.,  1993. 

Washington,  Trevor,  Chief,  Product  Assurance  Branch,  Air-to- 
Ground  Missile  Systems,  PEO  Tactical  Missiles,  Interview 
granted  June  22,  1995. 

Waters,  Melvin  E.,  Executive  Officer,  Total  Quality 
Management,  U.S.  Army  Missile  Command,  Interview  granted 
September  20,  1994. 

Watts,  Donna,  Program  Analyst,  Air-to-Ground  Missile 
Systems,  PEO  Tactical  Missiles,  Interview  granted  June  20, 
1995. 

White,  Laudrey,  Assistant  Project  Manager  (Technical 
Management) ,  Close  Combat  Anti-armor  Weapon  Systems,  PEO 
Tactical  Missiles,  Interview  granted  September  20,  1994; 

June  12,  1995. 

Witte,  W.  Stovall,  Lieutenant  Colonel,  U.S.  Army,  Assistant 
Project  Manager  (Production) ,  Air-to-Ground  Missile  Systems, 
PEO  Tactical  Missiles,  Interview  granted  June  19,  1995. 
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Woods,  W.  Max,  Professor  of  Statistics,  Operational  Research 
Department,  Naval  Postgraduate  School,  Interview  granted 
April  17,  1995. 
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